Re: [PR] [Opt] multi table func exec performance [doris]
HappenLee commented on PR #34090: URL: https://github.com/apache/doris/pull/34090#issuecomment-2080392723 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt] multi table func exec performance [doris]
github-actions[bot] commented on PR #34090: URL: https://github.com/apache/doris/pull/34090#issuecomment-2080394334 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](planner) Support `select constant from dual` syntax sugar [doris]
zclllyybb commented on PR #34200: URL: https://github.com/apache/doris/pull/34200#issuecomment-2080400212 what's this grammar for? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [Enhancement](external catalog) Added status reset when jdbc name mapping is abnormal (#33971)
This is an automated email from the ASF dual-hosted git repository. zykkk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 75cb520cbc5 [Enhancement](external catalog) Added status reset when jdbc name mapping is abnormal (#33971) 75cb520cbc5 is described below commit 75cb520cbc5b74a9ba47a4fdcfdaaeadcd33da35 Author: zy-kkk AuthorDate: Sat Apr 27 15:40:40 2024 +0800 [Enhancement](external catalog) Added status reset when jdbc name mapping is abnormal (#33971) --- .../datasource/mapping/IdentifierMapping.java | 82 +++--- 1 file changed, 57 insertions(+), 25 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/datasource/mapping/IdentifierMapping.java b/fe/fe-core/src/main/java/org/apache/doris/datasource/mapping/IdentifierMapping.java index cd121f2b630..363ef351152 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/datasource/mapping/IdentifierMapping.java +++ b/fe/fe-core/src/main/java/org/apache/doris/datasource/mapping/IdentifierMapping.java @@ -26,6 +26,8 @@ import com.fasterxml.jackson.databind.ObjectMapper; import com.google.common.collect.Lists; import com.google.common.collect.Maps; import com.google.common.collect.Sets; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; import java.util.Collections; import java.util.List; @@ -35,6 +37,7 @@ import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.atomic.AtomicBoolean; public abstract class IdentifierMapping { +private static final Logger LOG = LogManager.getLogger(IdentifierMapping.class); private final ObjectMapper mapper = new ObjectMapper(); private final ConcurrentHashMap localDBToRemoteDB = new ConcurrentHashMap<>(); @@ -179,51 +182,59 @@ public abstract class IdentifierMapping { } public String getRemoteDatabaseName(String localDbName) { -if (localDBToRemoteDB.isEmpty() || !localDBToRemoteDB.containsKey(localDbName)) { -loadDatabaseNamesIfNeeded(); -} -return localDBToRemoteDB.get(localDbName); +return getRequiredMapping(localDBToRemoteDB, localDbName, "database", this::loadDatabaseNamesIfNeeded, +localDbName); } public String getRemoteTableName(String localDbName, String localTableName) { String remoteDbName = getRemoteDatabaseName(localDbName); -if (localTableToRemoteTable.isEmpty() -|| !localTableToRemoteTable.containsKey(remoteDbName) -|| localTableToRemoteTable.get(remoteDbName) == null -|| localTableToRemoteTable.get(remoteDbName).isEmpty() -|| !localTableToRemoteTable.get(remoteDbName).containsKey(localTableName) -|| localTableToRemoteTable.get(remoteDbName).get(localTableName) == null) { -loadTableNamesIfNeeded(localDbName); -} - -return localTableToRemoteTable.get(remoteDbName).get(localTableName); +Map tableMap = localTableToRemoteTable.computeIfAbsent(remoteDbName, +k -> new ConcurrentHashMap<>()); +return getRequiredMapping(tableMap, localTableName, "table", () -> loadTableNamesIfNeeded(localDbName), +localTableName); } public Map getRemoteColumnNames(String localDbName, String localTableName) { String remoteDbName = getRemoteDatabaseName(localDbName); String remoteTableName = getRemoteTableName(localDbName, localTableName); -if (localColumnToRemoteColumn.isEmpty() -|| !localColumnToRemoteColumn.containsKey(remoteDbName) -|| localColumnToRemoteColumn.get(remoteDbName) == null -|| localColumnToRemoteColumn.get(remoteDbName).isEmpty() -|| !localColumnToRemoteColumn.get(remoteDbName).containsKey(remoteTableName) -|| localColumnToRemoteColumn.get(remoteDbName).get(remoteTableName) == null -|| localColumnToRemoteColumn.get(remoteDbName).get(remoteTableName).isEmpty()) { +ConcurrentHashMap> tableColumnMap += localColumnToRemoteColumn.computeIfAbsent(remoteDbName, k -> new ConcurrentHashMap<>()); +Map columnMap = tableColumnMap.computeIfAbsent(remoteTableName, k -> new ConcurrentHashMap<>()); +if (columnMap.isEmpty()) { +LOG.info("Column name mapping missing, loading column names for localDbName: {}, localTableName: {}", +localDbName, localTableName); loadColumnNamesIfNeeded(localDbName, localTableName); +columnMap = tableColumnMap.get(remoteTableName); } -return localColumnToRemoteColumn.get(remoteDbName).get(remoteTableName); +if (columnMap.isEmpty()) { +LOG.warn("No remote column found for localTableName: {}. Please refresh this catalog.", local
Re: [PR] [Enhancement](external catalog) Added status reset when jdbc name mapping is abnormal [doris]
zy-kkk merged PR #33971: URL: https://github.com/apache/doris/pull/33971 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](colocate) fix colocate join fail caused by subquery when nereids disabled [doris]
hqx871 commented on PR #34092: URL: https://github.com/apache/doris/pull/34092#issuecomment-2080412712 The master is much different from 1.2 which I used. And multi join with subquery work well after 17035, so just close the pr. The only left problem is the follow join should be bucket join but it choose to colocate join ``` select a.id,a.name,t1.id,t1.name,cid,cname from test_colo1 a join (select b.id,b.name,c.id as cid,c.name as cname from test_colo2 b left join test_colo3 c on b.id = c.id and b.name = c.name) t1 on a.id=t1.id and a.name= t1.name; ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](colocate) fix colocate join fail caused by subquery when nereids disabled [doris]
hqx871 closed pull request #34092: [fix](colocate) fix colocate join fail caused by subquery when nereids disabled URL: https://github.com/apache/doris/pull/34092 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](pipeline-load) fix no error url when data quality error and total rows is negative [doris]
liaoxin01 merged PR #34072: URL: https://github.com/apache/doris/pull/34072 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](pipeline-load) fix no error url when data quality error and total rows is negative (#34072)
This is an automated email from the ASF dual-hosted git repository. liaoxin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new d52a426002e [fix](pipeline-load) fix no error url when data quality error and total rows is negative (#34072) d52a426002e is described below commit d52a426002ec912516a592304146e5da2ac5997d Author: HHoflittlefish777 <77738092+hhoflittlefish...@users.noreply.github.com> AuthorDate: Sat Apr 27 16:21:16 2024 +0800 [fix](pipeline-load) fix no error url when data quality error and total rows is negative (#34072) --- .../runtime/stream_load/stream_load_executor.cpp | 47 + be/src/vec/sink/writer/vtablet_writer.cpp | 12 ++-- .../data/load_p0/stream_load/test_error_url.csv| 9 +++ .../load_p0/stream_load/test_pipeline_load.groovy | 2 +- .../stream_load/test_stream_load_error_url.groovy | 76 ++ .../test_partial_update_schema_change.groovy | 4 +- ...t_partial_update_schema_change_row_store.groovy | 4 +- 7 files changed, 113 insertions(+), 41 deletions(-) diff --git a/be/src/runtime/stream_load/stream_load_executor.cpp b/be/src/runtime/stream_load/stream_load_executor.cpp index 720c2e86898..58621c77a2a 100644 --- a/be/src/runtime/stream_load/stream_load_executor.cpp +++ b/be/src/runtime/stream_load/stream_load_executor.cpp @@ -78,38 +78,25 @@ Status StreamLoadExecutor::execute_plan_fragment(std::shared_ptrexec_env()->new_load_stream_mgr()->remove(ctx->id); ctx->commit_infos = std::move(state->tablet_commit_infos()); -if (status->ok()) { -ctx->number_total_rows = state->num_rows_load_total(); -ctx->number_loaded_rows = state->num_rows_load_success(); -ctx->number_filtered_rows = state->num_rows_load_filtered(); -ctx->number_unselected_rows = state->num_rows_load_unselected(); - -int64_t num_selected_rows = ctx->number_total_rows - ctx->number_unselected_rows; -if (!ctx->group_commit && num_selected_rows > 0 && -(double)ctx->number_filtered_rows / num_selected_rows > ctx->max_filter_ratio) { -// NOTE: Do not modify the error message here, for historical reasons, -// some users may rely on this error message. -*status = Status::DataQualityError("too many filtered rows"); -} -if (ctx->number_filtered_rows > 0 && !state->get_error_log_file_path().empty()) { -ctx->error_url = to_load_error_http_path(state->get_error_log_file_path()); -} +ctx->number_total_rows = state->num_rows_load_total(); +ctx->number_loaded_rows = state->num_rows_load_success(); +ctx->number_filtered_rows = state->num_rows_load_filtered(); +ctx->number_unselected_rows = state->num_rows_load_unselected(); +int64_t num_selected_rows = ctx->number_total_rows - ctx->number_unselected_rows; +if (!ctx->group_commit && num_selected_rows > 0 && +(double)ctx->number_filtered_rows / num_selected_rows > ctx->max_filter_ratio) { +// NOTE: Do not modify the error message here, for historical reasons, +// some users may rely on this error message. +*status = Status::DataQualityError("too many filtered rows"); +} +if (ctx->number_filtered_rows > 0 && !state->get_error_log_file_path().empty()) { +ctx->error_url = to_load_error_http_path(state->get_error_log_file_path()); +} -if (status->ok()) { - DorisMetrics::instance()->stream_receive_bytes_total->increment(ctx->receive_bytes); -DorisMetrics::instance()->stream_load_rows_total->increment( -ctx->number_loaded_rows); -} +if (status->ok()) { + DorisMetrics::instance()->stream_receive_bytes_total->increment(ctx->receive_bytes); + DorisMetrics::instance()->stream_load_rows_total->increment(ctx->number_loaded_rows); } else { -if (ctx->group_commit) { -ctx->number_total_rows = state->num_rows_load_total(); -ctx->number_loaded_rows = state->num_rows_load_success(); -ctx->number_filtered_rows = state->num_rows_load_filtered(); -ctx->number_unselected_rows = state->num_rows_load_unselected(); -if (ctx->number_filtered_rows > 0 && !state->get_error_log_file_path().empty()) { -ctx->error_url = to_load_error_http_path(state->get_error_log_file_path()); -} -} LOG(WARNING) << "fragment execute failed" << ", err_msg=" << status->to_string() << ", " << ctx->brief(); // cancel body_sink, make sender known it diff --git a/be/src/vec/sink/writer/vtablet_writer.cp
Re: [PR] [fix](profile) Fix print profile in be log [doris]
liaoxin01 commented on PR #34166: URL: https://github.com/apache/doris/pull/34166#issuecomment-2080414005 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](load) limit flush thread num proportional to CPU count [doris]
liaoxin01 merged PR #33325: URL: https://github.com/apache/doris/pull/33325 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [improve](load) limit flush thread num by CPU count (#33325)
This is an automated email from the ASF dual-hosted git repository. liaoxin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new a9040c8113b [improve](load) limit flush thread num by CPU count (#33325) a9040c8113b is described below commit a9040c8113b9f41ac480720ef98713d8dd550d5e Author: Kaijie Chen AuthorDate: Sat Apr 27 16:24:45 2024 +0800 [improve](load) limit flush thread num by CPU count (#33325) --- be/src/common/config.cpp| 3 +++ be/src/common/config.h | 3 +++ be/src/olap/memtable_flush_executor.cpp | 11 --- be/src/runtime/load_stream_mgr.cpp | 8 ++-- 4 files changed, 20 insertions(+), 5 deletions(-) diff --git a/be/src/common/config.cpp b/be/src/common/config.cpp index ea0e0bb3aad..0c257e48a8f 100644 --- a/be/src/common/config.cpp +++ b/be/src/common/config.cpp @@ -662,6 +662,9 @@ DEFINE_mInt64(storage_flood_stage_left_capacity_bytes, "1073741824"); // 1GB DEFINE_Int32(flush_thread_num_per_store, "6"); // number of thread for flushing memtable per store, for high priority load task DEFINE_Int32(high_priority_flush_thread_num_per_store, "6"); +// number of threads = min(flush_thread_num_per_store * num_store, +// max_flush_thread_num_per_cpu * num_cpu) +DEFINE_Int32(max_flush_thread_num_per_cpu, "4"); // config for tablet meta checkpoint DEFINE_mInt32(tablet_meta_checkpoint_min_new_rowsets_num, "10"); diff --git a/be/src/common/config.h b/be/src/common/config.h index 40d336d2308..e1ec93ff63f 100644 --- a/be/src/common/config.h +++ b/be/src/common/config.h @@ -715,6 +715,9 @@ DECLARE_mInt64(storage_flood_stage_left_capacity_bytes); // 1GB DECLARE_Int32(flush_thread_num_per_store); // number of thread for flushing memtable per store, for high priority load task DECLARE_Int32(high_priority_flush_thread_num_per_store); +// number of threads = min(flush_thread_num_per_store * num_store, +// max_flush_thread_num_per_cpu * num_cpu) +DECLARE_Int32(max_flush_thread_num_per_cpu); // config for tablet meta checkpoint DECLARE_mInt32(tablet_meta_checkpoint_min_new_rowsets_num); diff --git a/be/src/olap/memtable_flush_executor.cpp b/be/src/olap/memtable_flush_executor.cpp index db14b9acaee..b52a87dff07 100644 --- a/be/src/olap/memtable_flush_executor.cpp +++ b/be/src/olap/memtable_flush_executor.cpp @@ -202,15 +202,20 @@ void FlushToken::_flush_memtable(std::unique_ptr memtable_ptr, int32_t void MemTableFlushExecutor::init(int num_disk) { num_disk = std::max(1, num_disk); -size_t min_threads = std::max(1, config::flush_thread_num_per_store); -size_t max_threads = num_disk * min_threads; +int num_cpus = std::thread::hardware_concurrency(); +int min_threads = std::max(1, config::flush_thread_num_per_store); +int max_threads = num_cpus == 0 ? num_disk * min_threads +: std::min(num_disk * min_threads, + num_cpus * config::max_flush_thread_num_per_cpu); static_cast(ThreadPoolBuilder("MemTableFlushThreadPool") .set_min_threads(min_threads) .set_max_threads(max_threads) .build(&_flush_pool)); min_threads = std::max(1, config::high_priority_flush_thread_num_per_store); -max_threads = num_disk * min_threads; +max_threads = num_cpus == 0 ? num_disk * min_threads +: std::min(num_disk * min_threads, + num_cpus * config::max_flush_thread_num_per_cpu); static_cast(ThreadPoolBuilder("MemTableHighPriorityFlushThreadPool") .set_min_threads(min_threads) .set_max_threads(max_threads) diff --git a/be/src/runtime/load_stream_mgr.cpp b/be/src/runtime/load_stream_mgr.cpp index 7d9de12d078..1964cf257e3 100644 --- a/be/src/runtime/load_stream_mgr.cpp +++ b/be/src/runtime/load_stream_mgr.cpp @@ -37,9 +37,13 @@ LoadStreamMgr::LoadStreamMgr(uint32_t segment_file_writer_thread_num, : _num_threads(segment_file_writer_thread_num), _heavy_work_pool(heavy_work_pool), _light_work_pool(light_work_pool) { +uint32_t num_cpu = std::thread::hardware_concurrency(); +uint32_t thread_num = num_cpu == 0 ? segment_file_writer_thread_num + : std::min(segment_file_writer_thread_num, + num_cpu * config::max_flush_thread_num_per_cpu); static_cast(ThreadPoolBuilder("SegmentFileWriterThreadPool") - .set_min_threads(segment_file_writer_thread_num) - .set_max_threads(segment_file_writer_thread_num) + .set_min_threads(thread_num) +
Re: [PR] [improve](move-memtable) reduce default load stream per node to 2 for stream load [doris]
liaoxin01 merged PR #34065: URL: https://github.com/apache/doris/pull/34065 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [improve](move-memtable) reduce default load stream per node to 2 for stream load (#34065)
This is an automated email from the ASF dual-hosted git repository. liaoxin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new e24c7a5d216 [improve](move-memtable) reduce default load stream per node to 2 for stream load (#34065) e24c7a5d216 is described below commit e24c7a5d21671c7a756eda34bec11f8a072810f0 Author: Kaijie Chen AuthorDate: Sat Apr 27 16:29:48 2024 +0800 [improve](move-memtable) reduce default load stream per node to 2 for stream load (#34065) --- .../src/main/java/org/apache/doris/service/FrontendServiceImpl.java | 2 +- fe/fe-core/src/main/java/org/apache/doris/task/LoadTaskInfo.java| 2 +- fe/fe-core/src/main/java/org/apache/doris/task/StreamLoadTask.java | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java b/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java index c26712af0bd..d6029020834 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java +++ b/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java @@ -2073,7 +2073,7 @@ public class FrontendServiceImpl implements FrontendService.Iface { ctx.getSessionVariable().groupCommit = request.getGroupCommitMode(); try { HttpStreamParams httpStreamParams = initHttpStreamPlan(request, ctx); -int loadStreamPerNode = 20; +int loadStreamPerNode = 2; if (request.getStreamPerNode() > 0) { loadStreamPerNode = request.getStreamPerNode(); } diff --git a/fe/fe-core/src/main/java/org/apache/doris/task/LoadTaskInfo.java b/fe/fe-core/src/main/java/org/apache/doris/task/LoadTaskInfo.java index 8671933f758..8f641070c40 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/task/LoadTaskInfo.java +++ b/fe/fe-core/src/main/java/org/apache/doris/task/LoadTaskInfo.java @@ -126,7 +126,7 @@ public interface LoadTaskInfo { } default int getStreamPerNode() { -return 20; +return 2; } class ImportColumnDescs { diff --git a/fe/fe-core/src/main/java/org/apache/doris/task/StreamLoadTask.java b/fe/fe-core/src/main/java/org/apache/doris/task/StreamLoadTask.java index f663eb65913..94f3625fbc7 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/task/StreamLoadTask.java +++ b/fe/fe-core/src/main/java/org/apache/doris/task/StreamLoadTask.java @@ -89,7 +89,7 @@ public class StreamLoadTask implements LoadTaskInfo { private boolean enableProfile = false; private boolean memtableOnSinkNode = false; -private int streamPerNode = 20; +private int streamPerNode = 2; private byte enclose = 0; - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](planner) Support `select constant from dual` syntax sugar [doris]
morrySnow commented on PR #34200: URL: https://github.com/apache/doris/pull/34200#issuecomment-2080416508 add feature desc please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Testcases](fix) Fix some testcases [doris]
zclllyybb opened a new pull request, #34203: URL: https://github.com/apache/doris/pull/34203 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Testcases](fix) Fix some testcases [doris]
doris-robot commented on PR #34203: URL: https://github.com/apache/doris/pull/34203#issuecomment-2080416681 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Testcases](fix) Fix some testcases [doris]
zclllyybb commented on PR #34203: URL: https://github.com/apache/doris/pull/34203#issuecomment-2080416699 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](planner) Support `select constant from dual` syntax sugar [doris]
morrySnow commented on code in PR #34200: URL: https://github.com/apache/doris/pull/34200#discussion_r1581760829 ## regression-test/suites/query_p0/dual/dual.groovy: ## @@ -0,0 +1,70 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +suite('dual') { + +qt_sql 'select 1 from dual' +qt_sql 'select 1 from dual where 1' +qt_sql 'select 1 from dual where 1 = 1' +qt_sql 'select 1 from dual where 0' Review Comment: please add more complex test case. such as in cte, in subquery, in set operand, use with group by, window and other syntax -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt] multi table func exec performance [doris]
doris-robot commented on PR #34090: URL: https://github.com/apache/doris/pull/34090#issuecomment-2080417568 TPC-DS: Total hot run time: 188802 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 2b7fc023143f50be3828d34138b9273c932f3c81, data reload: false query1 906 362 350 350 query2 6187242024032403 query3 6648205 212 205 query4 22870 22012 21901 21901 query5 3853436 429 429 query6 278 191 191 191 query7 4538294 296 294 query8 243 191 188 188 query9 8438237023572357 query10 406 241 281 241 query11 15527 14880 14780 14780 query12 122 88 88 88 query13 1613355 370 355 query14 9224858686168586 query15 285 174 179 174 query16 8449260 263 260 query17 1861570 546 546 query18 2090271 266 266 query19 321 143 160 143 query20 94 81 79 79 query21 192 124 126 124 query22 4990480848074807 query23 33738 33405 33278 33278 query24 10878 289630092896 query25 594 378 363 363 query26 1176144 143 143 query27 2688310 309 309 query28 7493200519931993 query29 870 597 577 577 query30 247 150 152 150 query31 965 728 737 728 query32 94 51 53 51 query33 745 254 251 251 query34 1000475 472 472 query35 822 690 667 667 query36 1079944 948 944 query37 140 80 68 68 query38 3186302429902990 query39 1595155316441553 query40 195 131 132 131 query41 43 38 38 38 query42 106 98 96 96 query43 558 561 559 559 query44 1205738 735 735 query45 284 273 269 269 query46 1077724 707 707 query47 1937184718531847 query48 379 303 299 299 query49 832 408 399 399 query50 767 391 394 391 query51 6847659665826582 query52 109 92 93 92 query53 350 278 276 276 query54 302 249 241 241 query55 78 72 72 72 query56 248 234 228 228 query57 1202116211461146 query58 238 211 200 200 query59 3577336231843184 query60 270 243 249 243 query61 112 107 110 107 query62 635 440 432 432 query63 307 287 286 286 query64 8586721772457217 query65 3083314530363036 query66 947 336 335 335 query67 15846 14808 15004 14808 query68 10663 532 551 532 query69 631 301 299 299 query70 1381115111101110 query71 532 273 268 268 query72 9290263124562456 query73 1763319 318 318 query74 6615615261716152 query75 6139269826382638 query76 6700100010771000 query77 713 264 264 264 query78 11033 10496 10226 10226 query79 11751 510 519 510 query80 1767434 428 428 query81 492 232 221 221 query82 181 92 88 88 query83 209 168 159 159 query84 260 93 92 92 query85 1137316 257 257 query86 340 293 298 293 query87 3386311430863086 query88 5973233223422332 query89 501 367 371 367 query90 2345176 177 176 query91 121 96 98 96 query92 58 46 46 46 query93 6175498 500 498 query94 1601175 178 175 query95 397 306 288 288 query96 616 269 263 263 query97 3131292329202920 query98 229 212 215 212 query99 1062889 876 876 Total cold run time: 313757 ms Total hot run time: 188802 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
Re: [PR] [Testcases](fix) Fix some testcases [doris]
github-actions[bot] commented on PR #34203: URL: https://github.com/apache/doris/pull/34203#issuecomment-2080425487 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Testcases](fix) Fix some testcases [doris]
github-actions[bot] commented on PR #34203: URL: https://github.com/apache/doris/pull/34203#issuecomment-2080425479 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] Enhancement of round based function [doris]
superdiaodiao commented on issue #32755: URL: https://github.com/apache/doris/issues/32755#issuecomment-2080426672 May I ask the other three enhancement tasks assigned to anyone? If not, could you assign to me? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](memory) Fix thread context init in MacOS and not use memory tracker [doris]
xinyiZzz commented on PR #34125: URL: https://github.com/apache/doris/pull/34125#issuecomment-2080429384 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](memory) Fix thread context init in MacOS and not use memory tracker [doris]
github-actions[bot] commented on PR #34125: URL: https://github.com/apache/doris/pull/34125#issuecomment-2080430613 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](iceberg)support read iceberg complex type,iceberg.orc format and position delete. [doris]
hubgeter commented on PR #33935: URL: https://github.com/apache/doris/pull/33935#issuecomment-2080431902 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](iceberg)support read iceberg complex type,iceberg.orc format and position delete. [doris]
github-actions[bot] commented on PR #33935: URL: https://github.com/apache/doris/pull/33935#issuecomment-2080433218 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test) Fix some testcases [doris]
morrySnow commented on code in PR #34203: URL: https://github.com/apache/doris/pull/34203#discussion_r1581779625 ## regression-test/suites/ddl_p0/test_create_table_auto_partition.groovy: ## @@ -22,9 +22,9 @@ suite("test_create_table_auto_partition") { def testTable = "test_create_table_auto_partition_table" -sql "DROP TABLE IF EXISTS ${testTable}" +sql "DROP TABLE IF EXISTS test_create_table_auto_partition_table" Review Comment: Unexpected changes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](pipeline-load) fix no error url when data quality error and total rows is negative #34072 [doris]
liaoxin01 commented on PR #34204: URL: https://github.com/apache/doris/pull/34204#issuecomment-2080442294 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](pipeline-load) fix no error url when data quality error and total rows is negative #34072 [doris]
liaoxin01 merged PR #34204: URL: https://github.com/apache/doris/pull/34204 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [fix](pipeline-load) fix no error url when data quality error and total rows is negative (#34072) (#34204)
This is an automated email from the ASF dual-hosted git repository. liaoxin pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new cd1c9edd714 [fix](pipeline-load) fix no error url when data quality error and total rows is negative (#34072) (#34204) cd1c9edd714 is described below commit cd1c9edd71432a82c50226e335a699d2aa337ca8 Author: Xin Liao AuthorDate: Sat Apr 27 18:19:08 2024 +0800 [fix](pipeline-load) fix no error url when data quality error and total rows is negative (#34072) (#34204) Co-authored-by: HHoflittlefish777 <77738092+hhoflittlefish...@users.noreply.github.com> --- .../runtime/stream_load/stream_load_executor.cpp | 47 + be/src/vec/sink/writer/vtablet_writer.cpp | 12 ++-- .../data/load_p0/stream_load/test_error_url.csv| 9 +++ .../load_p0/stream_load/test_pipeline_load.groovy | 4 +- .../stream_load/test_stream_load_error_url.groovy | 76 ++ .../test_partial_update_schema_change.groovy | 4 +- ...t_partial_update_schema_change_row_store.groovy | 4 +- 7 files changed, 114 insertions(+), 42 deletions(-) diff --git a/be/src/runtime/stream_load/stream_load_executor.cpp b/be/src/runtime/stream_load/stream_load_executor.cpp index 720c2e86898..58621c77a2a 100644 --- a/be/src/runtime/stream_load/stream_load_executor.cpp +++ b/be/src/runtime/stream_load/stream_load_executor.cpp @@ -78,38 +78,25 @@ Status StreamLoadExecutor::execute_plan_fragment(std::shared_ptrexec_env()->new_load_stream_mgr()->remove(ctx->id); ctx->commit_infos = std::move(state->tablet_commit_infos()); -if (status->ok()) { -ctx->number_total_rows = state->num_rows_load_total(); -ctx->number_loaded_rows = state->num_rows_load_success(); -ctx->number_filtered_rows = state->num_rows_load_filtered(); -ctx->number_unselected_rows = state->num_rows_load_unselected(); - -int64_t num_selected_rows = ctx->number_total_rows - ctx->number_unselected_rows; -if (!ctx->group_commit && num_selected_rows > 0 && -(double)ctx->number_filtered_rows / num_selected_rows > ctx->max_filter_ratio) { -// NOTE: Do not modify the error message here, for historical reasons, -// some users may rely on this error message. -*status = Status::DataQualityError("too many filtered rows"); -} -if (ctx->number_filtered_rows > 0 && !state->get_error_log_file_path().empty()) { -ctx->error_url = to_load_error_http_path(state->get_error_log_file_path()); -} +ctx->number_total_rows = state->num_rows_load_total(); +ctx->number_loaded_rows = state->num_rows_load_success(); +ctx->number_filtered_rows = state->num_rows_load_filtered(); +ctx->number_unselected_rows = state->num_rows_load_unselected(); +int64_t num_selected_rows = ctx->number_total_rows - ctx->number_unselected_rows; +if (!ctx->group_commit && num_selected_rows > 0 && +(double)ctx->number_filtered_rows / num_selected_rows > ctx->max_filter_ratio) { +// NOTE: Do not modify the error message here, for historical reasons, +// some users may rely on this error message. +*status = Status::DataQualityError("too many filtered rows"); +} +if (ctx->number_filtered_rows > 0 && !state->get_error_log_file_path().empty()) { +ctx->error_url = to_load_error_http_path(state->get_error_log_file_path()); +} -if (status->ok()) { - DorisMetrics::instance()->stream_receive_bytes_total->increment(ctx->receive_bytes); -DorisMetrics::instance()->stream_load_rows_total->increment( -ctx->number_loaded_rows); -} +if (status->ok()) { + DorisMetrics::instance()->stream_receive_bytes_total->increment(ctx->receive_bytes); + DorisMetrics::instance()->stream_load_rows_total->increment(ctx->number_loaded_rows); } else { -if (ctx->group_commit) { -ctx->number_total_rows = state->num_rows_load_total(); -ctx->number_loaded_rows = state->num_rows_load_success(); -ctx->number_filtered_rows = state->num_rows_load_filtered(); -ctx->number_unselected_rows = state->num_rows_load_unselected(); -if (ctx->number_filtered_rows > 0 && !state->get_error_log_file_path().empty()) { -ctx->error_url = to_load_error_http_path(state->get_error_log_file_path()); -} -} LOG(WARNING) << "fragment execute failed" << ", err_msg=" << status->to_string() << ", " << ctx->brief(); // cancel body_sink, make sender
Re: [PR] [fix](pipeline-load) fix no error url when data quality error and total rows is negative #34072 [doris]
doris-robot commented on PR #34204: URL: https://github.com/apache/doris/pull/34204#issuecomment-2080442312 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](move-memtable) reduce default load stream per node to 2 for stream load #34065 [doris]
liaoxin01 commented on PR #34205: URL: https://github.com/apache/doris/pull/34205#issuecomment-2080442763 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](move-memtable) reduce default load stream per node to 2 for stream load #34065 [doris]
doris-robot commented on PR #34205: URL: https://github.com/apache/doris/pull/34205#issuecomment-2080442786 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve](move-memtable) reduce default load stream per node to 2 for stream load #34065 [doris]
liaoxin01 merged PR #34205: URL: https://github.com/apache/doris/pull/34205 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [improve](move-memtable) reduce default load stream per node to 2 for stream load (#34065) (#34205)
This is an automated email from the ASF dual-hosted git repository. liaoxin pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new 7ab425ee4b6 [improve](move-memtable) reduce default load stream per node to 2 for stream load (#34065) (#34205) 7ab425ee4b6 is described below commit 7ab425ee4b67ff20e6e7689f71bcfabad63519f8 Author: Xin Liao AuthorDate: Sat Apr 27 18:20:57 2024 +0800 [improve](move-memtable) reduce default load stream per node to 2 for stream load (#34065) (#34205) Co-authored-by: Kaijie Chen --- .../src/main/java/org/apache/doris/service/FrontendServiceImpl.java | 2 +- fe/fe-core/src/main/java/org/apache/doris/task/LoadTaskInfo.java| 2 +- fe/fe-core/src/main/java/org/apache/doris/task/StreamLoadTask.java | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java b/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java index 538a1f9893e..1d69c945805 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java +++ b/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java @@ -2085,7 +2085,7 @@ public class FrontendServiceImpl implements FrontendService.Iface { ctx.getSessionVariable().groupCommit = request.getGroupCommitMode(); try { HttpStreamParams httpStreamParams = initHttpStreamPlan(request, ctx); -int loadStreamPerNode = 20; +int loadStreamPerNode = 2; if (request.getStreamPerNode() > 0) { loadStreamPerNode = request.getStreamPerNode(); } diff --git a/fe/fe-core/src/main/java/org/apache/doris/task/LoadTaskInfo.java b/fe/fe-core/src/main/java/org/apache/doris/task/LoadTaskInfo.java index 8671933f758..8f641070c40 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/task/LoadTaskInfo.java +++ b/fe/fe-core/src/main/java/org/apache/doris/task/LoadTaskInfo.java @@ -126,7 +126,7 @@ public interface LoadTaskInfo { } default int getStreamPerNode() { -return 20; +return 2; } class ImportColumnDescs { diff --git a/fe/fe-core/src/main/java/org/apache/doris/task/StreamLoadTask.java b/fe/fe-core/src/main/java/org/apache/doris/task/StreamLoadTask.java index f663eb65913..94f3625fbc7 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/task/StreamLoadTask.java +++ b/fe/fe-core/src/main/java/org/apache/doris/task/StreamLoadTask.java @@ -89,7 +89,7 @@ public class StreamLoadTask implements LoadTaskInfo { private boolean enableProfile = false; private boolean memtableOnSinkNode = false; -private int streamPerNode = 20; +private int streamPerNode = 2; private byte enclose = 0; - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](cloud) refactor copy into make logic clear [doris]
HHoflittlefish777 commented on PR #34181: URL: https://github.com/apache/doris/pull/34181#issuecomment-2080443334 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](pipeline-load) fix no error url when data quality error and total rows is negative #34072 [doris]
github-actions[bot] commented on PR #34204: URL: https://github.com/apache/doris/pull/34204#issuecomment-2080444149 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test) Fix some testcases [doris]
zclllyybb commented on code in PR #34203: URL: https://github.com/apache/doris/pull/34203#discussion_r1581796601 ## regression-test/suites/ddl_p0/test_create_table_auto_partition.groovy: ## @@ -22,9 +22,9 @@ suite("test_create_table_auto_partition") { def testTable = "test_create_table_auto_partition_table" -sql "DROP TABLE IF EXISTS ${testTable}" +sql "DROP TABLE IF EXISTS test_create_table_auto_partition_table" Review Comment: all table name in testcases should avoid use variable. So I think this is proper -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] the alias way [doris]
HappenLee opened a new pull request, #34206: URL: https://github.com/apache/doris/pull/34206 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] the alias way [doris]
doris-robot commented on PR #34206: URL: https://github.com/apache/doris/pull/34206#issuecomment-2080448268 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](iceberg)support read iceberg complex type,iceberg.orc format and position delete. [doris]
doris-robot commented on PR #33935: URL: https://github.com/apache/doris/pull/33935#issuecomment-2080450724 TPC-H: Total hot run time: 41682 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 491a55f14798c329410a773bb505a0dc3173f648, data reload: false -- Round 1 -- q1 17611 441542884288 q2 2020193 201 193 q3 10435 131312121212 q4 10197 809 926 809 q5 7535279428432794 q6 221 133 134 133 q7 1062650 651 650 q8 9227220221402140 q9 9227684968766849 q10 9246394939773949 q11 448 249 247 247 q12 446 230 228 228 q13 18320 322231273127 q14 271 229 244 229 q15 521 497 485 485 q16 532 429 401 401 q17 990 676 723 676 q18 8472785077787778 q19 4412151415181514 q20 652 325 329 325 q21 5312418433883388 q22 349 267 275 267 Total cold run time: 117506 ms Total hot run time: 41682 ms - Round 2, with runtime_filter_mode=off - q1 4547442343844384 q2 380 278 262 262 q3 3157291829812918 q4 1906163115901590 q5 5521556755525552 q6 210 122 128 122 q7 2352199719651965 q8 3264345233963396 q9 8864891790218917 q10 4008374438423744 q11 585 500 498 498 q12 812 621 639 621 q13 15960 312231063106 q14 330 287 280 280 q15 530 485 512 485 q16 492 440 446 440 q17 1819153515241524 q18 789375257525 q19 1694154614981498 q20 2015180117561756 q21 15327 483747594759 q22 586 485 517 485 Total cold run time: 82252 ms Total hot run time: 55827 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement] (plsql) support show procedure status filters [doris]
xinyiZzz commented on code in PR #33264: URL: https://github.com/apache/doris/pull/33264#discussion_r1581790675 ## fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java: ## @@ -3486,7 +3488,19 @@ public LogicalPlan visitDropProcedure(DropProcedureContext ctx) { @Override public LogicalPlan visitShowProcedureStatus(ShowProcedureStatusContext ctx) { -return ParserUtils.withOrigin(ctx, () -> new ShowProcedureStatusCommand()); +Set whereExpr = Collections.emptySet(); +if (ctx.whereClause() != null) { +whereExpr = ExpressionUtils.extractConjunctionToSet( +getExpression(ctx.whereClause().booleanExpression())); +} + +if (ctx.valueExpression() != null) { +// parser allows only LIKE or WhereClause. Review Comment: additional comment: Mysql grammar: SHOW PROCEDURE STATUS [LIKE 'pattern' | WHERE expr] https://dev.mysql.com/doc/refman/8.0/en/show-procedure-status.html ## fe/fe-core/src/main/java/org/apache/doris/plsql/functions/DorisFunctionRegistry.java: ## @@ -89,15 +92,57 @@ private String qualified(String name) { return (ConnectContext.get().getDatabase() + "." + name).toUpperCase(); } +private String getDbName(long catalogId, long dbId) { +String dbName = ""; +CatalogIf catalog = Env.getCurrentEnv().getCatalogMgr().getCatalog(catalogId); +if (catalog != null) { +DatabaseIf db = catalog.getDbNullable(dbId); +if (db != null) { +dbName = db.getFullName(); +} +} +return dbName; +} + +public static boolean like(String str, String expr) { Review Comment: refer to `ShowFunctionsStmt`, `ShowEncryptKeysStmt`, `ShowTableCreationStmt` can rewrite as: ``` public boolean like(String str) { str = str.toLowerCase(); return str.matches(wild.replace(".", "\\.").replace("?", ".").replace("%", ".*").toLowerCase()); } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](iceberg)support read iceberg complex type,iceberg.orc format and position delete. [doris]
doris-robot commented on PR #33935: URL: https://github.com/apache/doris/pull/33935#issuecomment-2080452200 TeamCity be ut coverage result: Function Coverage: 35.50% (8926/25143) Line Coverage: 27.14% (73449/270652) Region Coverage: 26.32% (37948/144156) Branch Coverage: 23.11% (19327/83614) Coverage Report: http://coverage.selectdb-in.cc/coverage/491a55f14798c329410a773bb505a0dc3173f648_491a55f14798c329410a773bb505a0dc3173f648/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [feature](mtmv)Cherry pick single teble mv rewrite [doris]
seawinde opened a new pull request, #34207: URL: https://github.com/apache/doris/pull/34207 Support Single table query rewrite with out group by this is useful for complex filter or expresission the mv def and query is as following which can be query rewritten pr is: https://github.com/apache/doris/pull/34185 commit id is 0f95160a ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](mtmv)Cherry pick single teble mv rewrite [doris]
doris-robot commented on PR #34207: URL: https://github.com/apache/doris/pull/34207#issuecomment-2080453121 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](memory) Fix thread context init in MacOS and not use memory tracker [doris]
doris-robot commented on PR #34125: URL: https://github.com/apache/doris/pull/34125#issuecomment-2080453206 TPC-DS: Total hot run time: 183112 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit ce77f7e86d845dd33931925814630b4ba09ad69a, data reload: false query1 907 379 340 340 query2 6170232023392320 query3 6651203 204 203 query4 23692 21723 21816 21723 query5 3883422 414 414 query6 263 184 182 182 query7 4659297 310 297 query8 245 184 197 184 query9 8725238824032388 query10 418 244 237 237 query11 11746 11046 11501 11046 query12 121 90 89 89 query13 1758382 359 359 query14 10478 847076017601 query15 248 179 168 168 query16 8227261 257 257 query17 1924571 556 556 query18 2102280 274 274 query19 311 187 146 146 query20 90 89 82 82 query21 193 129 123 123 query22 5067481949164819 query23 33984 32940 32984 32940 query24 10632 291429012901 query25 582 361 367 361 query26 1092151 147 147 query27 2886317 317 317 query28 7205201419961996 query29 850 594 592 592 query30 248 146 148 146 query31 936 704 740 704 query32 80 51 51 51 query33 741 237 237 237 query34 1041475 486 475 query35 784 658 674 658 query36 1050901 863 863 query37 126 65 66 65 query38 3121306629572957 query39 1577152815201520 query40 196 121 122 121 query41 41 37 37 37 query42 102 92 93 92 query43 576 540 535 535 query44 1214729 741 729 query45 287 284 258 258 query46 1084745 703 703 query47 1925184218511842 query48 358 281 291 281 query49 833 387 385 385 query50 761 376 378 376 query51 6933666167716661 query52 97 90 86 86 query53 348 275 283 275 query54 310 229 223 223 query55 75 70 72 70 query56 234 218 215 215 query57 1221115411321132 query58 217 194 213 194 query59 3459315131963151 query60 259 225 227 225 query61 89 109 87 87 query62 643 453 438 438 query63 307 278 286 278 query64 8498720571807180 query65 3069310030573057 query66 903 326 333 326 query67 15473 15045 15040 15040 query68 6981533 546 533 query69 547 304 300 300 query70 1133109611271096 query71 503 265 265 265 query72 8263260824192419 query73 718 315 318 315 query74 6676607860516051 query75 4411262726602627 query76 43791049931 931 query77 698 279 258 258 query78 10859 10282 10185 10185 query79 8320519 519 519 query80 1309435 421 421 query81 492 219 224 219 query82 855 93 91 91 query83 201 185 167 167 query84 260 86 83 83 query85 1410259 266 259 query86 407 306 310 306 query87 3302310630823082 query88 5115231523152315 query89 519 372 383 372 query90 2076181 181 181 query91 126 93 95 93 query92 57 46 45 45 query93 6814510 515 510 query94 1213174 175 174 query95 386 296 288 288 query96 601 258 260 258 query97 3125293029682930 query98 231 225 212 212 query99 1302872 877 872 Total cold run time: 297614 ms Total hot run time: 183112 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [enhancement] (plsql) support show procedure status filters [doris]
xinyiZzz commented on code in PR #33264: URL: https://github.com/apache/doris/pull/33264#discussion_r1581803203 ## fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ShowProcedureStatusCommand.java: ## @@ -59,10 +68,53 @@ public ShowResultSetMetaData getMetaData() { return builder.build(); } +private void validateAndExtractFilters(StringBuilder dbFilter, StringBuilder procFilter) throws Exception { + +if (whereExpr.isEmpty()) { +return; +} +Set likeSet = whereExpr.stream().filter(Like.class::isInstance).collect(Collectors.toSet()); +Set equalTo = whereExpr.stream().filter(EqualTo.class::isInstance).collect(Collectors.toSet()); + +if (whereExpr.size() != likeSet.size() + equalTo.size()) { +throw new AnalysisException("Only support equalTo and Like filters."); +} + +equalTo.addAll(likeSet); + +Map filterMap = equalTo.stream() +.collect(Collectors.toMap(exp -> ((Slot) exp.child(0)).getName(), +exp -> ((Literal) exp.child(1)).getStringValue())); + +// we support filter on Db and Name and ProcedureName. +// But one column we can put only once and support conjuncts +for (Map.Entry elem : filterMap.entrySet()) { +String columnName = elem.getKey(); +if ((!columnName.equals("Db")) && (!columnName.equals("Name")) && (!columnName.equals("ProcedureName"))) { Review Comment: doris is case insensitive, so modify to `columnName.toLowerCase().equals("db")`, same everywhere else. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](iceberg)support read iceberg complex type,iceberg.orc format and position delete. [doris]
doris-robot commented on PR #33935: URL: https://github.com/apache/doris/pull/33935#issuecomment-2080454084 TPC-DS: Total hot run time: 186092 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 491a55f14798c329410a773bb505a0dc3173f648, data reload: false query1 935 362 349 349 query2 7010242723582358 query3 6640200 212 200 query4 26280 21164 21175 21164 query5 4119434 425 425 query6 273 195 165 165 query7 4584290 283 283 query8 245 194 189 189 query9 8447232523232323 query10 433 255 250 250 query11 14798 14335 14248 14248 query12 137 89 87 87 query13 1642382 377 377 query14 9787751466386638 query15 217 169 167 167 query16 7808266 269 266 query17 1285561 561 561 query18 1970284 276 276 query19 208 151 156 151 query20 92 93 88 88 query21 203 130 128 128 query22 4981475947974759 query23 33642 33130 33191 33130 query24 8165301829492949 query25 617 390 387 387 query26 708 159 156 156 query27 2248320 329 320 query28 5710202220072007 query29 866 625 613 613 query30 247 155 155 155 query31 1003733 760 733 query32 97 58 55 55 query33 611 267 254 254 query34 866 484 496 484 query35 775 698 690 690 query36 1097936 921 921 query37 105 67 69 67 query38 3163301529972997 query39 1577154215411541 query40 204 131 131 131 query41 44 41 46 41 query42 108 101 96 96 query43 558 554 546 546 query44 1071738 768 738 query45 279 270 242 242 query46 1076708 722 708 query47 1904184318711843 query48 369 301 293 293 query49 848 470 405 405 query50 764 372 393 372 query51 6857659767376597 query52 106 99 93 93 query53 351 279 272 272 query54 271 239 233 233 query55 78 73 72 72 query56 242 220 216 216 query57 1184110711491107 query58 217 197 196 196 query59 3490320532563205 query60 255 232 255 232 query61 94 110 90 90 query62 614 449 425 425 query63 300 278 270 270 query64 8198717871297129 query65 3131304330683043 query66 796 334 354 334 query67 15570 14915 15323 14915 query68 9622558 554 554 query69 582 306 297 297 query70 1382113110821082 query71 468 269 269 269 query72 8325260724582458 query73 1603325 325 325 query74 6562617860756075 query75 4526264126612641 query76 53481022954 954 query77 751 263 267 263 query78 11166 10348 10284 10284 query79 12529 521 509 509 query80 1698426 432 426 query81 497 223 222 222 query82 344 92 92 92 query83 202 164 173 164 query84 265 82 81 81 query85 867 265 257 257 query86 346 298 279 279 query87 3373306730483048 query88 4991231723152315 query89 516 381 370 370 query90 2024179 180 179 query91 121 99 100 99 query92 63 46 47 46 query93 6875523 509 509 query94 1334181 178 178 query95 1133109410961094 query96 624 261 255 255 query97 3131293929492939 query98 232 215 226 215 query99 1189881 858 858 Total cold run time: 305259 ms Total hot run time: 186092 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [enhancement] (plsql) support show procedure status filters [doris]
xinyiZzz commented on code in PR #33264: URL: https://github.com/apache/doris/pull/33264#discussion_r1581804376 ## fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/ShowProcedureStatusCommand.java: ## @@ -59,10 +68,53 @@ public ShowResultSetMetaData getMetaData() { return builder.build(); } +private void validateAndExtractFilters(StringBuilder dbFilter, StringBuilder procFilter) throws Exception { + +if (whereExpr.isEmpty()) { +return; +} +Set likeSet = whereExpr.stream().filter(Like.class::isInstance).collect(Collectors.toSet()); +Set equalTo = whereExpr.stream().filter(EqualTo.class::isInstance).collect(Collectors.toSet()); + +if (whereExpr.size() != likeSet.size() + equalTo.size()) { +throw new AnalysisException("Only support equalTo and Like filters."); +} + +equalTo.addAll(likeSet); + +Map filterMap = equalTo.stream() +.collect(Collectors.toMap(exp -> ((Slot) exp.child(0)).getName(), +exp -> ((Literal) exp.child(1)).getStringValue())); + +// we support filter on Db and Name and ProcedureName. +// But one column we can put only once and support conjuncts +for (Map.Entry elem : filterMap.entrySet()) { +String columnName = elem.getKey(); +if ((!columnName.equals("Db")) && (!columnName.equals("Name")) && (!columnName.equals("ProcedureName"))) { +throw new AnalysisException("Only supports filter Db, Name, ProcedureName with equalTo or LIKE"); +} +if (columnName.equals("Db")) { +if (dbFilter.length() != 0) { +throw new AnalysisException("Only supports filter Db only 1 time in where clause"); +} +dbFilter.append(elem.getValue()); Review Comment: column name and value both are case insensitive, so modify to `elem.getValue().toLowerCase` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](cloud) refactor copy into make logic clear [doris]
doris-robot commented on PR #34181: URL: https://github.com/apache/doris/pull/34181#issuecomment-2080455196 TPC-DS: Total hot run time: 182927 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit a019ad53f6a49a7040b543d86d43ff530106ff82, data reload: false query1 902 356 350 350 query2 6649237122982298 query3 6658204 202 202 query4 25738 21735 21965 21735 query5 4016422 419 419 query6 264 178 177 177 query7 4543291 285 285 query8 246 179 179 179 query9 8413235824022358 query10 392 252 239 239 query11 11562 11140 11197 11140 query12 118 90 87 87 query13 1650358 349 349 query14 8639866374257425 query15 263 178 175 175 query16 8242247 252 247 query17 1804539 538 538 query18 2101267 259 259 query19 325 142 160 142 query20 88 83 82 82 query21 188 117 120 117 query22 5003487347674767 query23 34126 33210 33129 33129 query24 10482 288129592881 query25 597 365 363 363 query26 1083149 142 142 query27 2189313 320 313 query28 6918200620262006 query29 860 591 604 591 query30 256 153 148 148 query31 980 709 711 709 query32 94 52 51 51 query33 742 245 238 238 query34 1056466 462 462 query35 822 673 658 658 query36 1026898 887 887 query37 104 65 66 65 query38 3124301930493019 query39 1596152215511522 query40 195 121 125 121 query41 40 39 39 39 query42 100 94 93 93 query43 560 543 531 531 query44 1209727 744 727 query45 264 268 252 252 query46 1067751 701 701 query47 1919184518381838 query48 374 295 290 290 query49 841 420 394 394 query50 786 389 369 369 query51 6777673766486648 query52 102 87 86 86 query53 336 269 284 269 query54 300 233 228 228 query55 78 69 69 69 query56 235 225 214 214 query57 1210113511321132 query58 215 194 202 194 query59 3343342131793179 query60 265 229 231 229 query61 91 91 87 87 query62 658 450 445 445 query63 308 283 281 281 query64 8427709171057091 query65 3100301730323017 query66 809 316 335 316 query67 15240 14866 15176 14866 query68 5301551 529 529 query69 482 295 302 295 query70 1184111511421115 query71 397 265 267 265 query72 7267276225492549 query73 714 318 322 318 query74 6443596259665962 query75 3355267526942675 query76 29181061966 966 query77 395 264 265 264 query78 10907 10365 10170 10170 query79 8296508 513 508 query80 2247451 445 445 query81 523 217 219 217 query82 152993 88 88 query83 321 169 172 169 query84 273 85 87 85 query85 1419318 309 309 query86 458 266 312 266 query87 3356312230573057 query88 5257232723322327 query89 553 367 371 367 query90 2005185 184 184 query91 134 204 97 97 query92 56 46 45 45 query93 6285507 500 500 query94 1132175 175 175 query95 365 291 295 291 query96 605 258 262 258 query97 3159290129272901 query98 233 213 225 213 query99 1195857 863 857 Total cold run time: 291472 ms Total hot run time: 182927 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [enhancement] (plsql) support show procedure status filters [doris]
xinyiZzz commented on PR #33264: URL: https://github.com/apache/doris/pull/33264#issuecomment-2080455854 `show procedure` already compatible with DBeaver, display procedures correctly.     -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](profile) Fix print profile in be log [doris]
yiguolei merged PR #34166: URL: https://github.com/apache/doris/pull/34166 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](profile) Fix print profile in be log (#34166)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new c3261f49314 [fix](profile) Fix print profile in be log (#34166) c3261f49314 is described below commit c3261f4931448620ba92b26807f6849d7a10e526 Author: zhiqiang AuthorDate: Sat Apr 27 19:20:40 2024 +0800 [fix](profile) Fix print profile in be log (#34166) --- be/src/pipeline/pipeline_fragment_context.cpp | 13 +++-- .../java/org/apache/doris/planner/StreamLoadPlanner.java| 2 ++ 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/be/src/pipeline/pipeline_fragment_context.cpp b/be/src/pipeline/pipeline_fragment_context.cpp index 03de71f3c8d..9cc71afd4d7 100644 --- a/be/src/pipeline/pipeline_fragment_context.cpp +++ b/be/src/pipeline/pipeline_fragment_context.cpp @@ -1505,6 +1505,13 @@ void PipelineFragmentContext::_close_fragment_instance() { Defer defer_op {[&]() { _is_fragment_instance_closed = true; }}; _runtime_profile->total_time_counter()->update(_fragment_watcher.elapsed_time()); static_cast(send_report(true)); +// Print profile content in info log is a tempoeray solution for stream load. +// Since stream load does not have someting like coordinator on FE, so +// backend can not report profile to FE, ant its profile can not be shown +// in the same way with other query. So we print the profile content to info log. +// Print profile content in log is harmful for log readability, info log will be +// full of profile content, and not just profile of stream load. +// We know it, but currently we do not have a cheap and good solution for this. if (_runtime_state->enable_profile()) { std::stringstream ss; // Compute the _local_time_percent before pretty_print the runtime_profile @@ -1513,8 +1520,10 @@ void PipelineFragmentContext::_close_fragment_instance() { // After add the operation, the print out like that: // UNION_NODE (id=0):(Active: 56.720us, non-child: 82.53%) // We can easily know the exec node execute time without child time consumed. -_runtime_state->runtime_profile()->compute_time_in_profile(); -_runtime_state->runtime_profile()->pretty_print(&ss); +for (const auto& runtime_profile_ptr : _runtime_state->pipeline_id_to_profile()) { +runtime_profile_ptr->pretty_print(&ss); +} + if (_runtime_state->load_channel_profile()) { _runtime_state->load_channel_profile()->pretty_print(&ss); } diff --git a/fe/fe-core/src/main/java/org/apache/doris/planner/StreamLoadPlanner.java b/fe/fe-core/src/main/java/org/apache/doris/planner/StreamLoadPlanner.java index e7bd2c80453..10dc8c3a432 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/planner/StreamLoadPlanner.java +++ b/fe/fe-core/src/main/java/org/apache/doris/planner/StreamLoadPlanner.java @@ -332,6 +332,7 @@ public class StreamLoadPlanner { queryOptions.setEnablePipelineXEngine(Config.enable_pipeline_load); queryOptions.setBeExecVersion(Config.be_exec_version); queryOptions.setIsReportSuccess(taskInfo.getEnableProfile()); +queryOptions.setEnableProfile(taskInfo.getEnableProfile()); queryOptions.setEnableMemtableOnSinkNode(enableMemtableOnSinkNode); params.setQueryOptions(queryOptions); TQueryGlobals queryGlobals = new TQueryGlobals(); @@ -569,6 +570,7 @@ public class StreamLoadPlanner { queryOptions.setEnablePipelineXEngine(Config.enable_pipeline_load); queryOptions.setBeExecVersion(Config.be_exec_version); queryOptions.setIsReportSuccess(taskInfo.getEnableProfile()); +queryOptions.setEnableProfile(taskInfo.getEnableProfile()); queryOptions.setEnableMemtableOnSinkNode(enableMemtableOnSinkNode); pipParams.setQueryOptions(queryOptions); - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] the alias way [doris]
yiguolei merged PR #34206: URL: https://github.com/apache/doris/pull/34206 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch 2.1.3-mtmv updated: the alias way (#34206)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch 2.1.3-mtmv in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/2.1.3-mtmv by this push: new cdf19a65ff4 the alias way (#34206) cdf19a65ff4 is described below commit cdf19a65ff4c975abac5f3585cb2d63237a4b8f3 Author: HappenLee AuthorDate: Sat Apr 27 20:08:55 2024 +0800 the alias way (#34206) --- fe/fe-core/src/main/java/org/apache/doris/rewrite/FunctionAlias.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/rewrite/FunctionAlias.java b/fe/fe-core/src/main/java/org/apache/doris/rewrite/FunctionAlias.java index 29cf2988cfd..878319ac563 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/rewrite/FunctionAlias.java +++ b/fe/fe-core/src/main/java/org/apache/doris/rewrite/FunctionAlias.java @@ -40,7 +40,7 @@ public final class FunctionAlias implements ExprRewriteRule { .put("date_sub", "days_sub").put("subdate", "days_sub").put("inet_ntoa", "ipv4_num_to_string") .put("inet_aton", "ipv4_string_to_num_or_null").put("inet6_ntoa", "ipv6_num_to_string") .put("inet6_aton", "ipv6_string_to_num_or_null").put("lcase", "lower").put("add_months", "months_add") -.put("current_timestamp", "now").put("localtime", "now").put("localtimestamp", "now").put("ifnull", "nvl") +.put("current_timestamp", "now").put("localtime", "now").put("localtimestamp", "now").put("nvl", "ifnull") .put("rand", "random").put("sha", "sha1").put("substr", "substring").put("ucase", "upper") .put("approx_count_distinct", "ndv").build(); - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](mtmv)Cherry pick single teble mv rewrite [doris]
yiguolei merged PR #34207: URL: https://github.com/apache/doris/pull/34207 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch 2.1.3-mtmv updated: [feature](mtmv)Support single table mv rewrite (#34185) (#34207)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch 2.1.3-mtmv in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/2.1.3-mtmv by this push: new f3df633df8d [feature](mtmv)Support single table mv rewrite (#34185) (#34207) f3df633df8d is described below commit f3df633df8d56812ee1e610b61b256661fc87150 Author: seawinde <149132972+seawi...@users.noreply.github.com> AuthorDate: Sat Apr 27 20:08:42 2024 +0800 [feature](mtmv)Support single table mv rewrite (#34185) (#34207) Support Single table query rewrite with out group by this is useful for complex filter or expresission the mv def and query is as following which can be query rewritten mv def: ``` select * from lineitem where l_comment like '%xx%' ``` query: ``` select l_linenumber, l_receiptdate from lineitem where l_comment like '%xx%' ``` Co-authored-by: zfr9527 --- .../org/apache/doris/nereids/rules/RuleSet.java| 8 + ... => MaterializedViewFilterProjectScanRule.java} | 20 ++- ...le.java => MaterializedViewFilterScanRule.java} | 19 +- ... => MaterializedViewProjectFilterScanRule.java} | 20 ++- ...e.java => MaterializedViewProjectScanRule.java} | 19 +- .../exploration/mv/MaterializedViewScanRule.java | 58 ++- .../nereids/rules/exploration/mv/StructInfo.java | 34 .../data/nereids_rules_p0/mv/scan/scan_table.out | 33 .../mv/dimension/dimension_1.groovy| 172 +++ .../mv/nested_mtmv/nested_mtmv.groovy | 2 +- .../nereids_rules_p0/mv/scan/scan_table.groovy | 191 + 11 files changed, 555 insertions(+), 21 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/RuleSet.java b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/RuleSet.java index d317b1e8738..7dcb7c0c008 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/RuleSet.java +++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/RuleSet.java @@ -44,11 +44,15 @@ import org.apache.doris.nereids.rules.exploration.mv.MaterializedViewFilterAggre import org.apache.doris.nereids.rules.exploration.mv.MaterializedViewFilterJoinRule; import org.apache.doris.nereids.rules.exploration.mv.MaterializedViewFilterProjectAggregateRule; import org.apache.doris.nereids.rules.exploration.mv.MaterializedViewFilterProjectJoinRule; +import org.apache.doris.nereids.rules.exploration.mv.MaterializedViewFilterProjectScanRule; +import org.apache.doris.nereids.rules.exploration.mv.MaterializedViewFilterScanRule; import org.apache.doris.nereids.rules.exploration.mv.MaterializedViewOnlyJoinRule; import org.apache.doris.nereids.rules.exploration.mv.MaterializedViewProjectAggregateRule; import org.apache.doris.nereids.rules.exploration.mv.MaterializedViewProjectFilterAggregateRule; import org.apache.doris.nereids.rules.exploration.mv.MaterializedViewProjectFilterJoinRule; +import org.apache.doris.nereids.rules.exploration.mv.MaterializedViewProjectFilterScanRule; import org.apache.doris.nereids.rules.exploration.mv.MaterializedViewProjectJoinRule; +import org.apache.doris.nereids.rules.exploration.mv.MaterializedViewProjectScanRule; import org.apache.doris.nereids.rules.expression.ExpressionOptimization; import org.apache.doris.nereids.rules.implementation.AggregateStrategies; import org.apache.doris.nereids.rules.implementation.LogicalAssertNumRowsToPhysicalAssertNumRows; @@ -243,6 +247,10 @@ public class RuleSet { .add(MaterializedViewFilterAggregateRule.INSTANCE) .add(MaterializedViewProjectFilterAggregateRule.INSTANCE) .add(MaterializedViewFilterProjectAggregateRule.INSTANCE) +.add(MaterializedViewFilterScanRule.INSTANCE) +.add(MaterializedViewFilterProjectScanRule.INSTANCE) +.add(MaterializedViewProjectScanRule.INSTANCE) +.add(MaterializedViewProjectFilterScanRule.INSTANCE) .build(); public static final List DPHYP_REORDER_RULES = ImmutableList.builder() diff --git a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewScanRule.java b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewFilterProjectScanRule.java similarity index 52% copy from fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewScanRule.java copy to fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewFilterProjectScanRule.java index c9624c176b9..7063030f24d 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewScanRule.java +++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewFilterProjectSca
(doris) branch branch-2.1 updated: 2.1.3-rc03
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new a6bf35efdf2 2.1.3-rc03 a6bf35efdf2 is described below commit a6bf35efdf27e6dadad0bcf1059c57f034666e00 Author: yiguolei AuthorDate: Sat Apr 27 20:54:06 2024 +0800 2.1.3-rc03 --- gensrc/script/gen_build_version.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gensrc/script/gen_build_version.sh b/gensrc/script/gen_build_version.sh index b8c9340eec8..dd68f4d93f7 100755 --- a/gensrc/script/gen_build_version.sh +++ b/gensrc/script/gen_build_version.sh @@ -31,7 +31,7 @@ build_version_prefix="doris" build_version_major=2 build_version_minor=1 build_version_patch=3 -build_version_rc_version="rc02" +build_version_rc_version="rc03" build_version="${build_version_prefix}-${build_version_major}.${build_version_minor}.${build_version_patch}-${build_version_rc_version}" - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) annotated tag 2.1.3-rc03 updated (a6bf35efdf2 -> 519e32b42a2)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to annotated tag 2.1.3-rc03 in repository https://gitbox.apache.org/repos/asf/doris.git *** WARNING: tag 2.1.3-rc03 was modified! *** from a6bf35efdf2 (commit) to 519e32b42a2 (tag) tagging a6bf35efdf27e6dadad0bcf1059c57f034666e00 (commit) replaces 2.1.3-rc02 by yiguolei on Sat Apr 27 20:54:29 2024 +0800 - Log - 2.1.3-rc03 --- No new revisions were added by this update. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](short circurt) fix return default value issue [doris]
felixwluo commented on PR #34186: URL: https://github.com/apache/doris/pull/34186#issuecomment-2080593403 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](short circurt) fix return default value issue [doris]
github-actions[bot] commented on PR #34186: URL: https://github.com/apache/doris/pull/34186#issuecomment-2080603137 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt] multi table func exec performance [doris]
HappenLee commented on PR #34090: URL: https://github.com/apache/doris/pull/34090#issuecomment-2080608474 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt] multi table func exec performance [doris]
github-actions[bot] commented on PR #34090: URL: https://github.com/apache/doris/pull/34090#issuecomment-2080620440 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](short circurt) fix return default value issue [doris]
doris-robot commented on PR #34186: URL: https://github.com/apache/doris/pull/34186#issuecomment-2080669618 TPC-H: Total hot run time: 41108 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 53af774126ff4658e86f0ad44291e357804e5129, data reload: false -- Round 1 -- q1 17693 434542194219 q2 2023187 190 187 q3 10539 125711741174 q4 10207 753 715 715 q5 7522277227462746 q6 217 127 128 127 q7 1040627 606 606 q8 9210213820862086 q9 9557694268336833 q10 9039396639033903 q11 453 240 234 234 q12 538 214 226 214 q13 17262 314132043141 q14 297 230 231 230 q15 504 462 464 462 q16 477 394 405 394 q17 973 659 712 659 q18 8370771878657718 q19 6117155415041504 q20 644 336 312 312 q21 5260419833703370 q22 342 274 289 274 Total cold run time: 118284 ms Total hot run time: 41108 ms - Round 2, with runtime_filter_mode=off - q1 4524438344084383 q2 361 277 278 277 q3 3220293030032930 q4 1922160416031603 q5 5458555855695558 q6 208 122 125 122 q7 2341198319971983 q8 3309345133893389 q9 8870900989718971 q10 4047371838603718 q11 592 503 499 499 q12 827 620 668 620 q13 16899 320931893189 q14 302 313 307 307 q15 533 502 489 489 q16 498 452 449 449 q17 1807148114821481 q18 7996770974957495 q19 7136161914971497 q20 2048176617351735 q21 13940 506049434943 q22 620 512 553 512 Total cold run time: 87458 ms Total hot run time: 56150 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Opt] multi table func exec performance [doris]
doris-robot commented on PR #34090: URL: https://github.com/apache/doris/pull/34090#issuecomment-2080694544 TPC-H: Total hot run time: 41203 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit e13775c6a0612e93baa38c895325f4cb59e85384, data reload: false -- Round 1 -- q1 17616 429442384238 q2 2011203 192 192 q3 10488 116312161163 q4 10204 817 762 762 q5 7517277227462746 q6 217 137 129 129 q7 1073617 637 617 q8 9225211320932093 q9 9354675567876755 q10 9019396338823882 q11 466 242 240 240 q12 469 222 232 222 q13 17377 319831233123 q14 284 233 248 233 q15 520 473 476 473 q16 506 423 400 400 q17 993 740 782 740 q18 8274775977227722 q19 6761153215651532 q20 655 320 324 320 q21 5185334941523349 q22 357 293 272 272 Total cold run time: 118571 ms Total hot run time: 41203 ms - Round 2, with runtime_filter_mode=off - q1 4594430443994304 q2 389 294 280 280 q3 3154291429192914 q4 1919165816801658 q5 5456541655515416 q6 214 121 123 121 q7 2353200120022001 q8 3223342334553423 q9 8804880789568807 q10 4100391237423742 q11 576 480 482 480 q12 763 615 600 600 q13 16588 308431033084 q14 320 276 299 276 q15 528 485 496 485 q16 504 428 435 428 q17 1786146915501469 q18 8053750273567356 q19 3038151415711514 q20 2027175117471747 q21 13343 498247164716 q22 551 485 479 479 Total cold run time: 82283 ms Total hot run time: 55300 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](short circurt) fix return default value issue [doris]
doris-robot commented on PR #34186: URL: https://github.com/apache/doris/pull/34186#issuecomment-2080695649 TPC-DS: Total hot run time: 186957 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 53af774126ff4658e86f0ad44291e357804e5129, data reload: false query1 2624382 356 356 query2 8473235523282328 query3 7849229 215 215 query4 25857 21172 21223 21172 query5 4107414 425 414 query6 259 187 194 187 query7 4587279 274 274 query8 240 179 181 179 query9 8712231422902290 query10 441 265 247 247 query11 14774 14150 14269 14150 query12 136 91 85 85 query13 1637368 368 368 query14 10557 820182238201 query15 276 169 167 167 query16 8255272 262 262 query17 2034565 561 561 query18 2144287 281 281 query19 344 151 152 151 query20 95 86 87 86 query21 197 130 130 130 query22 5022481548514815 query23 33929 33043 33102 33043 query24 9270290629412906 query25 590 416 389 389 query26 697 154 149 149 query27 2082315 317 315 query28 5735200020072000 query29 909 611 594 594 query30 293 152 155 152 query31 1006755 729 729 query32 92 54 55 54 query33 612 241 241 241 query34 877 464 486 464 query35 760 686 659 659 query36 1045874 886 874 query37 107 62 68 62 query38 3147303329892989 query39 1644151715321517 query40 200 127 124 124 query41 41 38 37 37 query42 105 93 98 93 query43 541 543 551 543 query44 1103719 730 719 query45 287 270 267 267 query46 1068719 699 699 query47 1939183318611833 query48 364 305 288 288 query49 956 407 392 392 query50 771 364 379 364 query51 6900671467536714 query52 95 89 91 89 query53 347 274 277 274 query54 272 229 230 229 query55 77 71 71 71 query56 244 222 212 212 query57 1201115411051105 query58 220 192 207 192 query59 3380324430323032 query60 253 230 235 230 query61 116 91 89 89 query62 621 452 446 446 query63 299 273 276 273 query64 8243713371057105 query65 3092301430563014 query66 819 336 350 336 query67 15698 14964 15322 14964 query68 5797538 533 533 query69 520 302 306 302 query70 1158115511511151 query71 476 265 265 265 query72 8222259624532453 query73 724 313 324 313 query74 6505614560246024 query75 3756265626972656 query76 41501002942 942 query77 655 258 263 258 query78 11191 10103 10081 10081 query79 8724530 513 513 query80 1621439 434 434 query81 546 223 216 216 query82 165491 88 88 query83 200 162 164 162 query84 266 83 82 82 query85 1379294 255 255 query86 458 318 275 275 query87 3276312331293123 query88 5076231122972297 query89 560 368 372 368 query90 2018184 182 182 query91 123 96 94 94 query92 55 45 50 45 query93 6855521 496 496 query94 1186181 181 181 query95 1097109910931093 query96 604 264 261 261 query97 3201295329582953 query98 236 220 214 214 query99 1225857 881 857 Total cold run time: 304181 ms Total hot run time: 186957 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [Opt] multi table func exec performance [doris]
doris-robot commented on PR #34090: URL: https://github.com/apache/doris/pull/34090#issuecomment-2080719150 TPC-DS: Total hot run time: 186926 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit e13775c6a0612e93baa38c895325f4cb59e85384, data reload: false query1 909 359 363 359 query2 6455241123822382 query3 6643205 216 205 query4 24375 21140 21188 21140 query5 4147408 420 408 query6 265 173 172 172 query7 4593298 287 287 query8 253 189 185 185 query9 8426232723352327 query10 445 240 250 240 query11 14905 14205 14170 14170 query12 140 88 86 86 query13 1644364 373 364 query14 10535 815681298129 query15 236 168 175 168 query16 7974257 258 257 query17 1787555 541 541 query18 2043286 264 264 query19 203 150 155 150 query20 96 93 89 89 query21 198 135 127 127 query22 5018486048464846 query23 33764 33024 33180 33024 query24 7463294130222941 query25 588 367 364 364 query26 701 151 148 148 query27 2203313 310 310 query28 5199201420212014 query29 839 610 596 596 query30 242 151 153 151 query31 966 745 726 726 query32 82 51 52 51 query33 553 256 245 245 query34 887 474 477 474 query35 775 680 684 680 query36 1023906 903 903 query37 102 68 65 65 query38 3175299030452990 query39 1589155715281528 query40 204 127 124 124 query41 43 38 39 38 query42 103 96 101 96 query43 593 539 538 538 query44 1077735 751 735 query45 280 257 263 257 query46 1083724 722 722 query47 1917184518381838 query48 367 302 290 290 query49 877 409 397 397 query50 774 381 392 381 query51 6909671167626711 query52 103 86 90 86 query53 349 282 273 273 query54 262 225 235 225 query55 78 68 72 68 query56 244 220 220 220 query57 1173114111401140 query58 219 216 201 201 query59 3439339232723272 query60 253 232 231 231 query61 90 90 88 88 query62 590 447 437 437 query63 306 275 278 275 query64 8231717471317131 query65 3094303830633038 query66 774 331 335 331 query67 15160 14965 15043 14965 query68 5208543 537 537 query69 477 299 297 297 query70 1196109811611098 query71 366 265 270 265 query72 7846273925452545 query73 699 325 329 325 query74 6573610360676067 query75 3384269526632663 query76 27451025997 997 query77 394 270 270 270 query78 10782 10223 10186 10186 query79 4672527 519 519 query80 2199442 447 442 query81 523 230 224 224 query82 114196 97 96 query83 302 172 174 172 query84 267 92 92 92 query85 1893325 309 309 query86 480 277 284 277 query87 3337305131483051 query88 4983234123362336 query89 498 375 374 374 query90 2059188 283 188 query91 125 95 96 95 query92 60 48 46 46 query93 5405511 507 507 query94 1306179 175 175 query95 400 304 291 291 query96 598 261 261 261 query97 3161295829752958 query98 240 216 217 216 query99 1205868 833 833 Total cold run time: 284557 ms Total hot run time: 186926 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [opt](mtmv) Add enable materialized view nest rewrite switch [doris]
yiguolei merged PR #34197: URL: https://github.com/apache/doris/pull/34197 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (c3261f49314 -> f5963c61d02)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from c3261f49314 [fix](profile) Fix print profile in be log (#34166) add f5963c61d02 [opt](mtmv) Add enable materialized view nest rewrite switch (#34197) No new revisions were added by this update. Summary of changes: .../java/org/apache/doris/nereids/memo/Memo.java | 6 -- .../apache/doris/nereids/memo/StructInfoMap.java | 21 +--- .../mv/AbstractMaterializedViewRule.java | 3 ++- .../exploration/mv/MaterializedViewUtils.java | 11 +-- .../java/org/apache/doris/qe/SessionVariable.java | 12 +++ .../doris/nereids/memo/StructInfoMapTest.java | 5 - .../mv/nested/nested_materialized_view.out | 23 +- .../mv/nested/nested_materialized_view.groovy | 16 +++ .../mv/nested_mtmv/nested_mtmv.groovy | 1 + 9 files changed, 71 insertions(+), 27 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](short circurt) fix return default value issue [doris]
felixwluo commented on PR #34186: URL: https://github.com/apache/doris/pull/34186#issuecomment-2080723676 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](short circurt) fix return default value issue [doris]
github-actions[bot] commented on PR #34186: URL: https://github.com/apache/doris/pull/34186#issuecomment-2080733530 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](function) add ip to IntegerType function [doris]
Mryange commented on PR #34198: URL: https://github.com/apache/doris/pull/34198#issuecomment-2080739567 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](planner) Support `select constant from dual` syntax sugar [doris]
zy-kkk commented on PR #34200: URL: https://github.com/apache/doris/pull/34200#issuecomment-2080759938 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](short circurt) fix return default value issue [doris]
doris-robot commented on PR #34186: URL: https://github.com/apache/doris/pull/34186#issuecomment-2080760109 TPC-H: Total hot run time: 40682 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 5536bf5f5b00dbcceb8848820be62cee793d48ff, data reload: false -- Round 1 -- q1 17622 446443764376 q2 2415185 193 185 q3 11059 121612111211 q4 10521 786 760 760 q5 7690280327252725 q6 219 134 135 134 q7 1079620 611 611 q8 9626216120742074 q9 9370676266566656 q10 8815378037523752 q11 460 238 254 238 q12 389 222 219 219 q13 18632 294829712948 q14 267 233 234 233 q15 519 483 477 477 q16 511 397 393 393 q17 952 691 684 684 q18 8085759075807580 q19 1622157015111511 q20 658 292 323 292 q21 5140395333453345 q22 350 278 287 278 Total cold run time: 116001 ms Total hot run time: 40682 ms - Round 2, with runtime_filter_mode=off - q1 4272418742444187 q2 384 261 269 261 q3 3021274527762745 q4 1869153615371536 q5 5307534453005300 q6 204 120 123 120 q7 2252189019111890 q8 3179330933443309 q9 8589861885838583 q10 3893373536783678 q11 583 484 503 484 q12 764 577 569 569 q13 16317 298929602960 q14 301 284 282 282 q15 509 498 481 481 q16 462 421 419 419 q17 1743148414571457 q18 7613756175137513 q19 1633155015101510 q20 1966175417571754 q21 5030482648844826 q22 582 502 511 502 Total cold run time: 70473 ms Total hot run time: 54366 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](short circurt) fix return default value issue [doris]
doris-robot commented on PR #34186: URL: https://github.com/apache/doris/pull/34186#issuecomment-2080784659 TPC-DS: Total hot run time: 187298 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 5536bf5f5b00dbcceb8848820be62cee793d48ff, data reload: false query1 918 361 345 345 query2 6470246622662266 query3 6651202 206 202 query4 23560 21118 21149 21118 query5 4162434 412 412 query6 257 174 171 171 query7 4584297 291 291 query8 255 207 188 188 query9 8314234323302330 query10 438 247 254 247 query11 14690 14280 14331 14280 query12 145 94 88 88 query13 1657377 388 377 query14 9457765475427542 query15 247 172 177 172 query16 8061251 269 251 query17 1864555 539 539 query18 2074274 273 273 query19 198 151 147 147 query20 106 92 92 92 query21 192 125 126 125 query22 5022486447784778 query23 33914 33366 33272 33272 query24 11845 296229602960 query25 663 367 364 364 query26 1782158 150 150 query27 3115321 329 321 query28 7692200520202005 query29 1040605 594 594 query30 301 148 151 148 query31 1001738 728 728 query32 87 51 55 51 query33 746 261 244 244 query34 1094482 480 480 query35 819 704 668 668 query36 1055924 926 924 query37 274 71 70 70 query38 3168303329842984 query39 1635155615301530 query40 276 123 127 123 query41 42 39 39 39 query42 106 96 93 93 query43 564 553 531 531 query44 1219732 756 732 query45 288 278 250 250 query46 1075702 720 702 query47 1907187118561856 query48 368 301 296 296 query49 1214425 418 418 query50 769 399 384 384 query51 6834686767246724 query52 106 96 93 93 query53 352 282 280 280 query54 319 247 249 247 query55 79 75 73 73 query56 249 230 231 230 query57 1222112211431122 query58 239 201 200 200 query59 3535327332603260 query60 262 251 243 243 query61 111 108 107 107 query62 639 438 437 437 query63 308 285 280 280 query64 9585721672017201 query65 3121305430373037 query66 1409348 361 348 query67 15633 14991 15105 14991 query68 8464563 576 563 query69 538 312 311 311 query70 1235110211481102 query71 580 267 277 267 query72 8095260224542454 query73 740 320 323 320 query74 6614613261566132 query75 4103266226832662 query76 5051989 986 986 query77 623 259 262 259 query78 11086 10139 10379 10139 query79 12693 520 509 509 query80 1813434 417 417 query81 491 227 216 216 query82 729 91 92 91 query83 199 168 164 164 query84 269 80 80 80 query85 1516258 254 254 query86 403 287 277 277 query87 3427306030763060 query88 5441232123122312 query89 544 370 379 370 query90 1977182 173 173 query91 120 93 95 93 query92 54 45 46 45 query93 7617526 510 510 query94 1225182 171 171 query95 1093110010911091 query96 617 264 263 263 query97 3161292129272921 query98 244 219 214 214 query99 1289848 851 848 Total cold run time: 313435 ms Total hot run time: 187298 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
Re: [PR] [Bug](join) do not short_circuit_for_probe for mark join [doris]
github-actions[bot] commented on PR #34170: URL: https://github.com/apache/doris/pull/34170#issuecomment-2080790922 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](function) add ip to IntegerType function [doris]
doris-robot commented on PR #34198: URL: https://github.com/apache/doris/pull/34198#issuecomment-2080797108 TPC-DS: Total hot run time: 185554 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit a45f439dff6161146ccb7111ecf162a8131c9604, data reload: false query1 910 359 356 356 query2 6465245223382338 query3 6660204 206 204 query4 24167 21192 21258 21192 query5 4213424 422 422 query6 280 191 181 181 query7 4588294 289 289 query8 253 194 187 187 query9 8490233923142314 query10 434 255 258 255 query11 14674 14143 14280 14143 query12 137 90 84 84 query13 1642376 377 376 query14 10872 730467776777 query15 239 173 174 173 query16 8006266 269 266 query17 1841582 548 548 query18 2052281 279 279 query19 204 156 163 156 query20 101 93 100 93 query21 196 126 130 126 query22 4995481248114811 query23 33872 33306 33211 33211 query24 10785 295529332933 query25 614 368 366 366 query26 1437155 150 150 query27 3026322 313 313 query28 7707201020002000 query29 920 625 591 591 query30 305 150 149 149 query31 977 740 737 737 query32 90 56 54 54 query33 752 253 254 253 query34 1056484 487 484 query35 854 688 666 666 query36 1052917 905 905 query37 128 65 67 65 query38 3146301929782978 query39 1601152115551521 query40 202 130 126 126 query41 43 37 38 37 query42 107 101 99 99 query43 545 536 519 519 query44 1263751 758 751 query45 290 281 269 269 query46 1078732 708 708 query47 1904184818851848 query48 363 295 308 295 query49 1082402 401 401 query50 769 388 376 376 query51 6827676867326732 query52 110 89 92 89 query53 355 282 281 281 query54 297 232 244 232 query55 77 73 74 73 query56 248 226 228 226 query57 1222117211361136 query58 224 206 204 204 query59 3321315931223122 query60 278 234 227 227 query61 90 101 89 89 query62 667 442 432 432 query63 304 286 281 281 query64 8626715071727150 query65 3121305130423042 query66 1416341 331 331 query67 15507 15067 15012 15012 query68 5264551 559 551 query69 493 316 308 308 query70 1146103111171031 query71 416 285 275 275 query72 7872263224322432 query73 697 322 325 322 query74 6561610960356035 query75 3483268026592659 query76 3424993 1019993 query77 443 278 268 268 query78 10789 10330 10216 10216 query79 7348519 572 519 query80 1854440 449 440 query81 542 217 226 217 query82 143798 89 89 query83 335 166 163 163 query84 274 82 83 82 query85 1622267 259 259 query86 501 310 307 307 query87 3258306931303069 query88 5192233523192319 query89 483 379 376 376 query90 2008184 185 184 query91 124 99 100 99 query92 65 47 47 47 query93 5511518 509 509 query94 1237179 176 176 query95 398 295 298 295 query96 651 269 265 265 query97 3124293529342934 query98 237 221 209 209 query99 1245878 872 872 Total cold run time: 298111 ms Total hot run time: 185554 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
[PR] [FIX](cases )fix ipv6 value for regress case [doris]
amorynan opened a new pull request, #34208: URL: https://github.com/apache/doris/pull/34208 ## Proposed changes 127.0.0.1 is not suitable for ipv6 Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [FIX](cases )fix ipv6 value for regress case [doris]
doris-robot commented on PR #34208: URL: https://github.com/apache/doris/pull/34208#issuecomment-2080811098 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [FIX](cases )fix ipv6 value for regress case [doris]
amorynan commented on PR #34208: URL: https://github.com/apache/doris/pull/34208#issuecomment-2080813448 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) 01/01: the alias way (#34206)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch 2.1.3-mtmv in repository https://gitbox.apache.org/repos/asf/doris.git commit 000d328930b46490184b8058aa88243bda6480cf Author: HappenLee AuthorDate: Sat Apr 27 20:08:55 2024 +0800 the alias way (#34206) --- fe/fe-core/src/main/java/org/apache/doris/rewrite/FunctionAlias.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/rewrite/FunctionAlias.java b/fe/fe-core/src/main/java/org/apache/doris/rewrite/FunctionAlias.java index 29cf2988cfd..878319ac563 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/rewrite/FunctionAlias.java +++ b/fe/fe-core/src/main/java/org/apache/doris/rewrite/FunctionAlias.java @@ -40,7 +40,7 @@ public final class FunctionAlias implements ExprRewriteRule { .put("date_sub", "days_sub").put("subdate", "days_sub").put("inet_ntoa", "ipv4_num_to_string") .put("inet_aton", "ipv4_string_to_num_or_null").put("inet6_ntoa", "ipv6_num_to_string") .put("inet6_aton", "ipv6_string_to_num_or_null").put("lcase", "lower").put("add_months", "months_add") -.put("current_timestamp", "now").put("localtime", "now").put("localtimestamp", "now").put("ifnull", "nvl") +.put("current_timestamp", "now").put("localtime", "now").put("localtimestamp", "now").put("nvl", "ifnull") .put("rand", "random").put("sha", "sha1").put("substr", "substring").put("ucase", "upper") .put("approx_count_distinct", "ndv").build(); - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch 2.1.3-mtmv updated (cdf19a65ff4 -> 000d328930b)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch 2.1.3-mtmv in repository https://gitbox.apache.org/repos/asf/doris.git discard cdf19a65ff4 the alias way (#34206) discard f3df633df8d [feature](mtmv)Support single table mv rewrite (#34185) (#34207) new 000d328930b the alias way (#34206) This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (cdf19a65ff4) \ N -- N -- N refs/heads/2.1.3-mtmv (000d328930b) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../org/apache/doris/nereids/rules/RuleSet.java| 8 - .../mv/MaterializedViewFilterProjectScanRule.java | 45 - .../mv/MaterializedViewFilterScanRule.java | 44 - .../mv/MaterializedViewProjectFilterScanRule.java | 45 - .../mv/MaterializedViewProjectScanRule.java| 44 - .../exploration/mv/MaterializedViewScanRule.java | 58 +-- .../nereids/rules/exploration/mv/StructInfo.java | 34 .../data/nereids_rules_p0/mv/scan/scan_table.out | 33 .../mv/dimension/dimension_1.groovy| 172 --- .../mv/nested_mtmv/nested_mtmv.groovy | 2 +- .../nereids_rules_p0/mv/scan/scan_table.groovy | 191 - 11 files changed, 5 insertions(+), 671 deletions(-) delete mode 100644 fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewFilterProjectScanRule.java delete mode 100644 fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewFilterScanRule.java delete mode 100644 fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewProjectFilterScanRule.java delete mode 100644 fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/mv/MaterializedViewProjectScanRule.java delete mode 100644 regression-test/data/nereids_rules_p0/mv/scan/scan_table.out delete mode 100644 regression-test/suites/nereids_rules_p0/mv/scan/scan_table.groovy - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [FIX](cases )fix ipv6 value for regress case [doris]
github-actions[bot] commented on PR #34208: URL: https://github.com/apache/doris/pull/34208#issuecomment-2080827150 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [FIX](cases )fix ipv6 value for regress case [doris]
github-actions[bot] commented on PR #34208: URL: https://github.com/apache/doris/pull/34208#issuecomment-2080827067 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](planner) Support `select constant from dual` syntax sugar [doris]
doris-robot commented on PR #34200: URL: https://github.com/apache/doris/pull/34200#issuecomment-2080834800 TPC-H: Total hot run time: 41066 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit b6b72407c44717623a7bdb6a22a14da223de8a42, data reload: false -- Round 1 -- q1 17606 428441904190 q2 2015191 187 187 q3 10462 122712161216 q4 10193 797 796 796 q5 7526270626752675 q6 225 129 135 129 q7 1021632 634 632 q8 9225209520642064 q9 9405674867516748 q10 8855390839133908 q11 460 242 244 242 q12 445 223 234 223 q13 17747 325031583158 q14 269 239 245 239 q15 516 480 473 473 q16 546 419 411 411 q17 944 658 720 658 q18 8289783876307630 q19 6480154714621462 q20 652 327 311 311 q21 5131343141833431 q22 361 285 283 283 Total cold run time: 118373 ms Total hot run time: 41066 ms - Round 2, with runtime_filter_mode=off - q1 4558442244584422 q2 377 286 282 282 q3 3177293129092909 q4 1955168815591559 q5 5346555655355535 q6 221 126 128 126 q7 2319202319231923 q8 3295343034193419 q9 8796887988468846 q10 4046379937473747 q11 597 505 492 492 q12 789 629 647 629 q13 16917 312231723122 q14 326 272 291 272 q15 524 482 475 475 q16 493 448 431 431 q17 1806154215021502 q18 7964760073007300 q19 2781159015521552 q20 2040180317521752 q21 14574 475247954752 q22 558 507 503 503 Total cold run time: 83459 ms Total hot run time: 0 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](planner) Support `select constant from dual` syntax sugar [doris]
doris-robot commented on PR #34200: URL: https://github.com/apache/doris/pull/34200#issuecomment-2080858740 TPC-DS: Total hot run time: 186793 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit b6b72407c44717623a7bdb6a22a14da223de8a42, data reload: false query1 918 363 367 363 query2 6466247522502250 query3 6648201 209 201 query4 23273 21640 21256 21256 query5 4148436 432 432 query6 269 201 168 168 query7 4576288 285 285 query8 238 194 192 192 query9 8440233323032303 query10 423 260 252 252 query11 14751 14216 14249 14216 query12 140 89 87 87 query13 1643384 382 382 query14 9767842176517651 query15 221 172 172 172 query16 7968275 264 264 query17 1719582 567 567 query18 2026292 276 276 query19 209 160 156 156 query20 93 87 86 86 query21 199 135 133 133 query22 4991490047694769 query23 34481 33266 6 33266 query24 6759306229592959 query25 570 392 390 390 query26 704 165 165 165 query27 2030316 328 316 query28 3633204120212021 query29 904 649 620 620 query30 228 159 156 156 query31 978 765 745 745 query32 93 54 55 54 query33 502 264 262 262 query34 892 501 484 484 query35 778 699 682 682 query36 1034874 918 874 query37 111 69 69 69 query38 3143299730122997 query39 1575154115361536 query40 199 135 131 131 query41 42 41 40 40 query42 106 100 98 98 query43 560 543 556 543 query44 1103739 749 739 query45 300 273 266 266 query46 1070758 707 707 query47 1923183718671837 query48 374 313 291 291 query49 798 399 409 399 query50 768 396 382 382 query51 6894682766976697 query52 106 94 97 94 query53 348 287 290 287 query54 265 237 233 233 query55 76 71 71 71 query56 247 220 221 220 query57 1199113211311131 query58 240 213 220 213 query59 3314324232023202 query60 265 236 242 236 query61 92 92 89 89 query62 578 431 440 431 query63 310 282 284 282 query64 8182721671907190 query65 3152299830242998 query66 806 347 352 347 query67 15648 15021 15198 15021 query68 9592562 576 562 query69 559 311 325 311 query70 1302102911461029 query71 495 278 287 278 query72 8070266824412441 query73 1514325 329 325 query74 6523610060826082 query75 4421272526922692 query76 5030960 1001960 query77 689 272 270 270 query78 11259 10325 10109 10109 query79 12421 525 531 525 query80 1731455 435 435 query81 496 224 222 222 query82 604 95 92 92 query83 230 169 170 169 query84 270 85 85 85 query85 849 271 284 271 query86 343 294 299 294 query87 3334309531143095 query88 5013234523152315 query89 534 370 381 370 query90 2064188 188 188 query91 127 100 98 98 query92 59 54 48 48 query93 6812525 514 514 query94 1023198 178 178 query95 398 310 298 298 query96 613 259 259 259 query97 3083296129652961 query98 241 223 220 220 query99 1208883 865 865 Total cold run time: 297413 ms Total hot run time: 186793 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [refactor](cloud) refactor copy into make logic clear [doris]
HHoflittlefish777 commented on PR #34181: URL: https://github.com/apache/doris/pull/34181#issuecomment-2080868526 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](MethodName) fix method issue [doris]
yiguolei merged PR #34178: URL: https://github.com/apache/doris/pull/34178 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [Fix](MethodName) fix method issue #34178
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new ea80da9edfb [Fix](MethodName) fix method issue #34178 ea80da9edfb is described below commit ea80da9edfb30bdc577b1759b1f5dddc74a3709e Author: GoGoWen <82132356+gogo...@users.noreply.github.com> AuthorDate: Sat Apr 27 23:15:37 2024 +0800 [Fix](MethodName) fix method issue #34178 --- fe/fe-core/src/main/java/org/apache/doris/clone/TabletSchedCtx.java | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/clone/TabletSchedCtx.java b/fe/fe-core/src/main/java/org/apache/doris/clone/TabletSchedCtx.java index 3a190ef11c2..9d54ce2dc5f 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/clone/TabletSchedCtx.java +++ b/fe/fe-core/src/main/java/org/apache/doris/clone/TabletSchedCtx.java @@ -775,7 +775,7 @@ public class TabletSchedCtx implements Comparable { boolean allCatchup = true; for (Replica replica : furtherRepairs) { -if (checkFurthurRepairFinish(replica, visibleVersion)) { +if (checkFurtherRepairFinish(replica, visibleVersion)) { replica.setNeedFurtherRepair(false); replica.setFurtherRepairWatermarkTxnTd(-1); } else { @@ -858,7 +858,7 @@ public class TabletSchedCtx implements Comparable { setDest(chosenReplica.getBackendId(), chosenReplica.getPathHash()); } -private boolean checkFurthurRepairFinish(Replica replica, long version) { +private boolean checkFurtherRepairFinish(Replica replica, long version) { if (replica.getVersion() < version || replica.getLastFailedVersion() > 0) { return false; } @@ -1209,7 +1209,7 @@ public class TabletSchedCtx implements Comparable { // change from prepare to committed or visible, this replica will be fall behind and be removed // in REDUNDANT detection. // -boolean isCatchup = checkFurthurRepairFinish(replica, partition.getVisibleVersion()); +boolean isCatchup = checkFurtherRepairFinish(replica, partition.getVisibleVersion()); replica.incrFurtherRepairCount(); if (isCatchup || replica.getLeftFurtherRepairCount() <= 0) { replica.setNeedFurtherRepair(false); - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](test) fix some external test cases [doris]
morningman opened a new pull request, #34209: URL: https://github.com/apache/doris/pull/34209 ## Proposed changes Fix some test cases and enable `test_information_schema_external` suite ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test) fix some external test cases [doris]
doris-robot commented on PR #34209: URL: https://github.com/apache/doris/pull/34209#issuecomment-2080896767 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test) fix some external test cases [doris]
morningman merged PR #34209: URL: https://github.com/apache/doris/pull/34209 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [fix](test) fix some external test cases (#34209)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-2.1 by this push: new 45556686ea4 [fix](test) fix some external test cases (#34209) 45556686ea4 is described below commit 45556686ea45d024e3759a4dc933d7c6b6e2fed4 Author: Mingyu Chen AuthorDate: Sat Apr 27 23:25:33 2024 +0800 [fix](test) fix some external test cases (#34209) Fix some test cases and enable `test_information_schema_external` suite --- .../hive/test_information_schema_external.out | 244 ++--- ...est_external_catalog_iceberg_hadoop_catalog.out | 9 +- .../test_external_catalog_iceberg_partition.out| 46 ++-- .../pipeline/p0/conf/regression-conf.groovy| 1 - .../hive/test_information_schema_external.groovy | 42 ++-- .../hive/test_prepare_hive_data_in_case.groovy | 2 +- .../paimon/paimon_base_types.groovy| 4 +- 7 files changed, 174 insertions(+), 174 deletions(-) diff --git a/regression-test/data/external_table_p0/hive/test_information_schema_external.out b/regression-test/data/external_table_p0/hive/test_information_schema_external.out index 05a9f7b6093..199b9e0b123 100644 --- a/regression-test/data/external_table_p0/hive/test_information_schema_external.out +++ b/regression-test/data/external_table_p0/hive/test_information_schema_external.out @@ -1,17 +1,17 @@ -- This file is automatically generated. You should know what you did if you want to edit this -- !schemata_1 -- -test_information_schema_external default utf8utf8_general_ci \N +test_information_schema_external_hive2 default utf8mb4 utf8mb4_0900_bin \N NO -- !schemata_2 -- -internal info_schema_ext_db_1utf8utf8_general_ci \N +internal info_schema_ext_db_1utf8mb4 utf8mb4_0900_bin\N NO -- !schemata_3 -- -internal info_schema_ext_db_2utf8utf8_general_ci \N +internal info_schema_ext_db_2utf8mb4 utf8mb4_0900_bin\N NO -- !schemata_4 -- -- !schemata_5 -- -internal info_schema_ext_db_2utf8utf8_general_ci \N +internal info_schema_ext_db_2utf8mb4 utf8mb4_0900_bin\N NO -- !schemata_6 -- @@ -30,101 +30,101 @@ internal info_schema_ext_db abcdid 1 \N NO int \N \N 10 0 \N \N \N int(11) UNI internal info_schema_ext_db abcdname2 \N YES varchar 2147483643 8589934572 \N \N \N \N \N string 2147483643 \N \N \N -- !columns_3 -- -test_information_schema_external tpch1_parquet customer c_acctbal 6 \N YES decimal \N \N 12 2 \N \N \N decimalv3(12, 2) 12 2 \N \N -test_information_schema_external tpch1_parquet customer c_address 3 \N YES varchar 2147483643 8589934572 \N \N \N \N \N string 2147483643 \N \N \N -test_information_schema_external tpch1_parquet customer c_comment 8 \N YES varchar 2147483643 8589934572 \N \N \N \N \N string 2147483643 \N \N \N -test_information_schema_external tpch1_parquet customer c_custkey 1 \N YES int \N \N 10 0 \N \N \N int(11) 10 0 \N \N -test_information_schema_external tpch1_parquet customer c_mktsegment7 \N YES varchar 2147483643 8589934572 \N \N \N \N \N string 2147483643 \N \N \N -test_information_schema_external tpch1_parquet customerc_name 2 \N YES varchar 2147483643 8589934572 \N \N \N \N \N string 2147483643 \N \N \N -test_information_schema_external tpch1_parquet customer c_nationkey 4 \N YES int \N \N 10 0 \N \N \N int(11) 10 0 \N \N -test_information_schema_external tpch1_parquet customerc_phone 5 \N YES varchar 2147483643 8589934572 \N \N \N \N \N string 2147483643 \N \N \N +test_information_schema_external_hive2 tpch1_parquet customer c_acctbal 6 \N YES
Re: [PR] [refactor](cloud) refactor copy into make logic clear [doris]
doris-robot commented on PR #34181: URL: https://github.com/apache/doris/pull/34181#issuecomment-2080911682 TPC-H: Total hot run time: 40355 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 8c9d4492bcb48b881853b04a966024bf20831123, data reload: false -- Round 1 -- q1 17647 473243354335 q2 2676184 185 184 q3 10934 128012651265 q4 10217 774 780 774 q5 7645273727662737 q6 220 129 130 129 q7 1022609 603 603 q8 9267210520562056 q9 9117677566266626 q10 8690367537393675 q11 466 243 234 234 q12 476 215 217 215 q13 17761 296129192919 q14 265 243 242 242 q15 505 470 474 470 q16 525 383 380 380 q17 953 697 702 697 q18 8033750075017500 q19 5659153714521452 q20 679 310 318 310 q21 5110327938883279 q22 351 273 282 273 Total cold run time: 118218 ms Total hot run time: 40355 ms - Round 2, with runtime_filter_mode=off - q1 4363423141984198 q2 373 282 260 260 q3 2973276827242724 q4 1830158815711571 q5 5331534553145314 q6 209 119 120 119 q7 2280187618951876 q8 3190334033513340 q9 8546856085958560 q10 3894369936953695 q11 581 481 491 481 q12 782 575 586 575 q13 17363 296830112968 q14 307 264 274 264 q15 508 485 480 480 q16 457 406 420 406 q17 1764147914861479 q18 7796749874857485 q19 1636155815321532 q20 1957177017591759 q21 4938475948074759 q22 577 520 496 496 Total cold run time: 71655 ms Total hot run time: 54341 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement] (plsql) support show procedure status filters [doris]
Vallishp commented on PR #33264: URL: https://github.com/apache/doris/pull/33264#issuecomment-2080916656 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](pipeline) Trace pipeline schedule (part II) - illustrate demo [doris]
zclllyybb commented on PR #31301: URL: https://github.com/apache/doris/pull/31301#issuecomment-2080919826 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test) fix trino hive connector unstable test case [doris]
morningman closed pull request #34158: [fix](test) fix trino hive connector unstable test case URL: https://github.com/apache/doris/pull/34158 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](test) fix some external test cases [doris]
morningman opened a new pull request, #34210: URL: https://github.com/apache/doris/pull/34210 ## Proposed changes Fix some test cases and enable `test_information_schema_external` suite ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test) fix some external test cases [doris]
doris-robot commented on PR #34210: URL: https://github.com/apache/doris/pull/34210#issuecomment-2080921839 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org