Re: [PR] [feature](nereids) Support SELECT * REPLACE clause [doris]
doris-robot commented on PR #38680: URL: https://github.com/apache/doris/pull/38680#issuecomment-2262197319 TPC-DS: Total hot run time: 169621 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit ab90698d95aa8f7b2569ccc70cf0e6199ba0f266, data reload: false query1 926 375 373 373 query2 6474168616891686 query3 6687216 221 216 query4 19972 17601 17362 17362 query5 4322535 523 523 query6 290 194 151 151 query7 4601298 301 298 query8 243 198 195 195 query9 8510235423602354 query10 455 289 267 267 query11 10492 999299219921 query12 135 91 87 87 query13 1656387 374 374 query14 11318 795785677957 query15 210 161 170 161 query16 6833452 444 444 query17 954 584 552 552 query18 1359294 289 289 query19 199 153 148 148 query20 99 87 89 87 query21 206 102 101 101 query22 4229388139283881 query23 33806 33257 32946 32946 query24 10375 308230333033 query25 712 401 400 400 query26 1722159 156 156 query27 2959286 277 277 query28 6918197219581958 query29 1103440 442 440 query30 291 153 154 153 query31 942 753 770 753 query32 103 58 58 58 query33 706 317 321 317 query34 924 474 487 474 query35 849 747 739 739 query36 984 863 871 863 query37 258 78 79 78 query38 2868275827702758 query39 847 800 830 800 query40 281 116 113 113 query41 48 48 46 46 query42 125 105 107 105 query43 459 432 433 432 query44 1278724 714 714 query45 207 173 178 173 query46 1085806 802 802 query47 1768169116921691 query48 363 303 295 295 query49 1171426 424 424 query50 891 436 441 436 query51 6893676067006700 query52 109 90 99 90 query53 254 191 176 176 query54 638 450 454 450 query55 79 75 75 75 query56 288 247 273 247 query57 1180104910571049 query58 275 269 263 263 query59 2659245223342334 query60 294 270 272 270 query61 97 97 107 97 query62 889 663 657 657 query63 220 192 202 192 query64 5992191619011901 query65 3187308330833083 query66 1378326 332 326 query67 15159 14815 14665 14665 query68 4547568 579 568 query69 486 314 300 300 query70 1113107210111011 query71 441 284 270 270 query72 7762269025242524 query73 766 328 336 328 query74 6005561556755615 query75 3636275427602754 query76 2806133414191334 query77 499 341 317 317 query78 9403897188918891 query79 2471562 534 534 query80 1168500 500 500 query81 587 226 216 216 query82 744 130 129 129 query83 238 169 172 169 query84 275 79 83 79 query85 2056303 295 295 query86 473 326 277 277 query87 3276310630853085 query88 3831239424182394 query89 391 289 287 287 query90 1910192 190 190 query91 126 99 99 99 query92 70 54 55 54 query93 2464617 610 610 query94 921 283 294 283 query95 390 270 277 270 query96 612 279 280 279 query97 3242302030693020 query98 230 237 199 199 query99 1698133013041304 Total cold run time: 268441 ms Total hot run time: 169621 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [fix](merge-on-write) segcompaction should process delete bitmap if necessary [doris]
zhannngchen merged PR #38369: URL: https://github.com/apache/doris/pull/38369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](merge-on-write) segcompaction should process delete bitmap if necessary (#38369)
This is an automated email from the ASF dual-hosted git repository.
zhangchen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new c03b564d6a1 [fix](merge-on-write) segcompaction should process delete
bitmap if necessary (#38369)
c03b564d6a1 is described below
commit c03b564d6a1ae477d58d3370fb5e9f9c60e92cc2
Author: zhannngchen <48427519+zhannngc...@users.noreply.github.com>
AuthorDate: Thu Aug 1 15:00:28 2024 +0800
[fix](merge-on-write) segcompaction should process delete bitmap if
necessary (#38369)
## Proposed changes
Issue Number: close #xxx
When loading data to a unique key table with sequence column, some data
in current load job might be marked as delete due to a lower sequence
value.
If there's many segments in such load job, segcompaction might be
triggered, which don't process the delete bitmap currently, will cause
data correctness issue
For example:
1. we have 4 segments in current load job initially, and due to seq
column, some rows are marked as deleted
2. after segcompaction, if we don't process the delete bitmap, it's
content is still corresponding to the old segment layout, and row
7,14,15 is not mark deleted correctly on new generated segment 1.
3. in this PR, we convert old delete bitmap to fit new segment layout,
it use similar way as base/cumulative compaction to convert delete
bitmaps on old layout to new one, but the rowid conversion is simpler
---
be/src/olap/merger.cpp| 19 +-
be/src/olap/merger.h | 6 +-
be/src/olap/rowset/beta_rowset_writer.cpp | 34 +-
be/src/olap/rowset/beta_rowset_writer.h | 3 +-
be/src/olap/rowset/segcompaction.cpp | 67 ++-
be/src/olap/rowset/segcompaction.h| 13 +
be/src/olap/simple_rowid_conversion.h | 84 +++
be/src/vec/olap/vertical_block_reader.cpp | 1 +
be/test/common/status_test.cpp| 2 -
be/test/olap/segcompaction_mow_test.cpp | 890 ++
10 files changed, 1100 insertions(+), 19 deletions(-)
diff --git a/be/src/olap/merger.cpp b/be/src/olap/merger.cpp
index ad70241ad87..87792db93a6 100644
--- a/be/src/olap/merger.cpp
+++ b/be/src/olap/merger.cpp
@@ -336,16 +336,12 @@ Status Merger::vertical_compact_one_group(
}
// for segcompaction
-Status Merger::vertical_compact_one_group(int64_t tablet_id, ReaderType
reader_type,
- const TabletSchema& tablet_schema,
bool is_key,
- const std::vector&
column_group,
- vectorized::RowSourcesBuffer*
row_source_buf,
- vectorized::VerticalBlockReader&
src_block_reader,
- segment_v2::SegmentWriter&
dst_segment_writer,
- int64_t max_rows_per_segment,
Statistics* stats_output,
- uint64_t* index_size, KeyBoundsPB&
key_bounds) {
-// build tablet reader
-VLOG_NOTICE << "vertical compact one group, max_rows_per_segment=" <<
max_rows_per_segment;
+Status Merger::vertical_compact_one_group(
+int64_t tablet_id, ReaderType reader_type, const TabletSchema&
tablet_schema, bool is_key,
+const std::vector& column_group,
vectorized::RowSourcesBuffer* row_source_buf,
+vectorized::VerticalBlockReader& src_block_reader,
+segment_v2::SegmentWriter& dst_segment_writer, Statistics*
stats_output,
+uint64_t* index_size, KeyBoundsPB& key_bounds, SimpleRowIdConversion*
rowid_conversion) {
// TODO: record_rowids
vectorized::Block block = tablet_schema.create_block(column_group);
size_t output_rows = 0;
@@ -362,6 +358,9 @@ Status Merger::vertical_compact_one_group(int64_t
tablet_id, ReaderType reader_t
"failed to write block when merging
rowsets of tablet " +
std::to_string(tablet_id));
+if (is_key && rowid_conversion != nullptr) {
+
rowid_conversion->add(src_block_reader.current_block_row_locations());
+}
output_rows += block.rows();
block.clear_column_data();
}
diff --git a/be/src/olap/merger.h b/be/src/olap/merger.h
index 7513c90fbd1..cb05162b3bc 100644
--- a/be/src/olap/merger.h
+++ b/be/src/olap/merger.h
@@ -23,6 +23,7 @@
#include "io/io_common.h"
#include "olap/iterators.h"
#include "olap/rowset/rowset_fwd.h"
+#include "olap/simple_rowid_conversion.h"
#include "olap/tablet_fwd.h"
namespace doris {
@@ -82,8 +83,9 @@ public:
vectorized::Row
Re: [PR] [Improvement](mow) Make calculate delete bitmap timeout configurable [doris]
github-actions[bot] commented on PR #38668: URL: https://github.com/apache/doris/pull/38668#issuecomment-2262198467 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](fe) Fix the default value of ReplacePartitionClause.isStrictRange [doris]
w41ter opened a new pull request, #38688: URL: https://github.com/apache/doris/pull/38688 During ReplacePartitionClause.analyze(), the default value of isStrictRange is true. However, ReplacePartitionClause instances constructed by internal code do not call analyze(), so their isStrictRange value is incorrect (e.g., INSERT INTO ... OVERWRITE). This PR handle the relevant properties when constructing ReplacePartitionClause. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](fe) Fix the default value of ReplacePartitionClause.isStrictRange [doris]
w41ter commented on PR #38688: URL: https://github.com/apache/doris/pull/38688#issuecomment-2262203279 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](fe) Fix the default value of ReplacePartitionClause.isStrictRange [doris]
doris-robot commented on PR #38688: URL: https://github.com/apache/doris/pull/38688#issuecomment-2262202968 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](case) fix jsonb-load output [doris]
amorynan commented on PR #38661: URL: https://github.com/apache/doris/pull/38661#issuecomment-2262205875 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](nereids) Support SELECT * REPLACE clause [doris]
doris-robot commented on PR #38680: URL: https://github.com/apache/doris/pull/38680#issuecomment-2262206044 ClickBench: Total hot run time: 29.9 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit ab90698d95aa8f7b2569ccc70cf0e6199ba0f266, data reload: false query1 0.040.040.04 query2 0.070.040.03 query3 0.230.050.04 query4 1.680.070.06 query5 0.560.500.48 query6 1.150.720.72 query7 0.020.010.01 query8 0.050.040.04 query9 0.570.500.52 query10 0.560.560.56 query11 0.140.110.11 query12 0.150.120.12 query13 0.620.600.59 query14 0.770.800.79 query15 0.910.860.86 query16 0.350.350.35 query17 1.051.010.97 query18 0.220.210.21 query19 1.881.781.74 query20 0.020.010.01 query21 15.41 0.780.66 query22 3.897.321.17 query23 17.99 1.301.35 query24 2.290.230.22 query25 0.190.080.08 query26 0.320.220.22 query27 0.450.230.24 query28 13.16 1.000.96 query29 12.52 3.353.32 query30 0.260.060.06 query31 2.840.420.40 query32 3.230.490.48 query33 2.962.962.92 query34 15.46 4.274.26 query35 4.314.324.33 query36 0.690.490.48 query37 0.200.160.16 query38 0.160.160.15 query39 0.040.040.04 query40 0.160.130.12 query41 0.110.040.05 query42 0.060.050.04 query43 0.050.040.04 Total cold run time: 107.79 s Total hot run time: 29.9 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](nereids) Support SELECT * REPLACE clause [doris]
morrySnow commented on code in PR #38680:
URL: https://github.com/apache/doris/pull/38680#discussion_r1699544183
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindExpression.java:
##
@@ -633,7 +639,31 @@ private Plan
bindProject(MatchingContext> ctx) {
if (!excepts.isEmpty()) {
slots = Utils.filterImmutableList(slots, slot ->
!boundExcepts.get().contains(slot));
}
-boundProjections.addAll(slots);
+if (!replaces.isEmpty()) {
+final Map replaceMap = new
HashMap<>();
+final Set replaced = new HashSet<>();
+for (NamedExpression replace : boundReplaces.get()) {
+Preconditions.checkArgument(replace instanceof Alias);
+Alias alias = (Alias) replace;
+replaceMap.put(alias.getName(), alias);
+}
+
+for (Slot slot : slots) {
+if (replaceMap.containsKey(slot.getName())) {
Review Comment:
what will happen if query like `select * replace ('123' as c1) from t, t t2`
when t has columns `c1`?
i think we should thrown exception same as `select c1 from t, t t2`;
so the better way to do it may be we should use alias name as UnboundSlot,
and use them to bind with project's child's output. and then we use
`org.apache.doris.nereids.util.ExpressionUtils#replace` to replace project list.
##
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##
@@ -3009,10 +3012,23 @@ protected LogicalPlan withSelectQuerySpecification(
throw new ParseException("only column name is
supported in except clause", selectColumnCtx);
}
UnboundStar star = new UnboundStar(ImmutableList.of());
-project = new LogicalProject<>(ImmutableList.of(star),
expressions, isDistinct, aggregate);
+project = new LogicalProject<>(
+ImmutableList.of(star), expressions,
ImmutableList.of(), isDistinct, aggregate);
+} else if (selectColumnCtx.REPLACE() != null) {
Review Comment:
`else if` not a good way. if we support use expect + replace, we should use
`if`. if we do not support use them togather, we should throw exception.
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalProject.java:
##
Review Comment:
update hashCode, equals and toString
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](err-msg) Add detailed column and schema info when failed to create a column iterator [doris]
doris-robot commented on PR #38689: URL: https://github.com/apache/doris/pull/38689#issuecomment-2262211297 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](statisitcs)Disable external catalog auto analyze by default. [doris]
Jibing-Li commented on PR #38674: URL: https://github.com/apache/doris/pull/38674#issuecomment-2262211359 run p1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [2.0](dependencies)upgrade dependencies [doris]
CalvinKirs merged PR #38671: URL: https://github.com/apache/doris/pull/38671 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [2.0](dependencies)upgrade dependencies (#38671)
This is an automated email from the ASF dual-hosted git repository.
kirs pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new aced7cf1ec6 [2.0](dependencies)upgrade dependencies (#38671)
aced7cf1ec6 is described below
commit aced7cf1ec606c0bee696dc6f020e16c223aead4
Author: Calvin Kirs
AuthorDate: Thu Aug 1 15:08:24 2024 +0800
[2.0](dependencies)upgrade dependencies (#38671)
## Proposed changes
Issue Number: close #xxx
---
fe/pom.xml | 18 --
1 file changed, 12 insertions(+), 6 deletions(-)
diff --git a/fe/pom.xml b/fe/pom.xml
index 3cb5af10e6e..234d9fc3bcd 100644
--- a/fe/pom.xml
+++ b/fe/pom.xml
@@ -260,14 +260,15 @@ under the License.
2.18.0
2.0.6
4.0.2
-4.1.94.Final
+
+4.1.104.Final
3.10.6.Final
2.1
1.34.0
-1.58.0
-3.38.0
+1.60.1
+3.43.0
3.24.3
@@ -275,7 +276,7 @@ under the License.
com.google.protobuf:protoc:${protoc.artifact.version}
io.grpc:protoc-gen-grpc-java:${grpc-java.version}
3.1.5
-1.1.10.1
+1.1.10.5
1.11-8
1.0.1
6.7.2
@@ -283,7 +284,7 @@ under the License.
4.9.3
3.4.0
2.0
-1.1.0.Final
+2.0.1.Final
0.2.3
3.4.0
6.4.5
@@ -342,6 +343,7 @@ under the License.
3.0.0
2.3.2
+0.27
0.7.0-incubating
3.4.4
@@ -417,7 +419,11 @@ under the License.
opentelemetry-sdk
${opentelemetry.version}
-
+
+io.airlift
+aircompressor
+${aircompressor.version}
+
io.opentelemetry
opentelemetry-exporter-otlp
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [enhancement](err-msg) Add detailed column and schema info when failed to create a column iterator [doris]
TangSiyang2001 opened a new pull request, #38689: URL: https://github.com/apache/doris/pull/38689 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.1] Picks "[opt](merge-on-write) Skip the alignment process of some rowsets in partial update #38487" [doris]
doris-robot commented on PR #38682: URL: https://github.com/apache/doris/pull/38682#issuecomment-2262214552 TeamCity be ut coverage result: Function Coverage: 36.43% (9254/25402) Line Coverage: 27.98% (75676/270417) Region Coverage: 26.79% (38891/145191) Branch Coverage: 23.51% (19735/83932) Coverage Report: http://coverage.selectdb-in.cc/coverage/e38eedb85a2627e9e2342e661797354d17cb906d_e38eedb85a2627e9e2342e661797354d17cb906d/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [regression](kerberos)add hive with kerberos write back case [doris]
wsjz commented on PR #38647: URL: https://github.com/apache/doris/pull/38647#issuecomment-2262217769 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Json) fix some cast issue [doris]
doris-robot commented on PR #38683: URL: https://github.com/apache/doris/pull/38683#issuecomment-2262220483 TPC-H: Total hot run time: 41330 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit b06f3e3fe1ecb4d99c8304c9fd5442f2e18607ff, data reload: false -- Round 1 -- q1 17695 408040094009 q2 2017202 199 199 q3 10461 132613131313 q4 10164 776 892 776 q5 7635298929422942 q6 220 138 138 138 q7 1019620 609 609 q8 9435190119391901 q9 8651662765636563 q10 8753383438053805 q11 434 249 253 249 q12 431 229 230 229 q13 17761 294629422942 q14 268 251 247 247 q15 523 486 493 486 q16 525 393 385 385 q17 954 918 895 895 q18 7996732771937193 q19 1488121312021202 q20 565 330 335 330 q21 5290464546374637 q22 346 298 280 280 Total cold run time: 112631 ms Total hot run time: 41330 ms - Round 2, with runtime_filter_mode=off - q1 4026399740283997 q2 338 230 226 226 q3 2986300630993006 q4 1960201019721972 q5 5609549554415441 q6 233 130 133 130 q7 2161180218411802 q8 3336341433633363 q9 8708869387888693 q10 3939401539693969 q11 536 453 451 451 q12 774 564 604 564 q13 15206 314531453145 q14 301 270 270 270 q15 527 507 489 489 q16 466 421 414 414 q17 1770173117231723 q18 8074784476317631 q19 1725174317061706 q20 2067187418531853 q21 5756553754605460 q22 522 462 468 462 Total cold run time: 71020 ms Total hot run time: 56767 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [enhancement](err-msg) Add detailed column and schema info when failed to create a column iterator [doris]
github-actions[bot] commented on PR #38689: URL: https://github.com/apache/doris/pull/38689#issuecomment-226365 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [draft](cloud-merge) Support shadow tablet to do cumulative compaction in cloud mode [doris]
Lchangliang commented on PR #37293: URL: https://github.com/apache/doris/pull/37293#issuecomment-226622 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](nereids) Support SELECT * REPLACE clause [doris]
macroguo-ghy commented on code in PR #38680:
URL: https://github.com/apache/doris/pull/38680#discussion_r1699559515
##
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##
@@ -3009,10 +3012,23 @@ protected LogicalPlan withSelectQuerySpecification(
throw new ParseException("only column name is
supported in except clause", selectColumnCtx);
}
UnboundStar star = new UnboundStar(ImmutableList.of());
-project = new LogicalProject<>(ImmutableList.of(star),
expressions, isDistinct, aggregate);
+project = new LogicalProject<>(
+ImmutableList.of(star), expressions,
ImmutableList.of(), isDistinct, aggregate);
+} else if (selectColumnCtx.REPLACE() != null) {
Review Comment:
ok, I will fix it. I think we should support combine `REPLACE` and `EXCEPT`.
I see ck support this syntax,
https://clickhouse.com/docs/en/sql-reference/statements/select#modifier-combinations
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](hdfs-fs)The cache expiration should explicitly release the held fs [doris]
doris-robot commented on PR #38610: URL: https://github.com/apache/doris/pull/38610#issuecomment-2262223114 TPC-H: Total hot run time: 41306 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit d9e3536ad4eebdd65bf77a485a0e78f21fdf8d6e, data reload: false -- Round 1 -- q1 17601 408840434043 q2 2022197 197 197 q3 10464 132513121312 q4 10172 873 911 873 q5 7622292229492922 q6 220 136 137 136 q7 1029606 592 592 q8 9425192219291922 q9 8673657965606560 q10 8780384837843784 q11 431 246 241 241 q12 424 220 220 220 q13 17791 290029172900 q14 271 254 250 250 q15 519 495 491 491 q16 514 393 391 391 q17 949 918 861 861 q18 7919735072037203 q19 1430121311981198 q20 554 323 336 323 q21 5258461747404617 q22 338 270 272 270 Total cold run time: 112406 ms Total hot run time: 41306 ms - Round 2, with runtime_filter_mode=off - q1 4019398839923988 q2 321 225 223 223 q3 2975297330412973 q4 1975196820071968 q5 5503549954245424 q6 232 131 128 128 q7 2098173518461735 q8 3309338033213321 q9 8643863786588637 q10 3999396640613966 q11 547 442 449 442 q12 721 589 575 575 q13 15761 309230823082 q14 302 267 275 267 q15 530 478 502 478 q16 459 416 422 416 q17 1755174217161716 q18 8109774975937593 q19 1703173217241724 q20 2055183618111811 q21 5669538554625385 q22 564 470 458 458 Total cold run time: 71249 ms Total hot run time: 56310 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test) Modify SQLServer image to custom hub (#38515) [doris]
doris-robot commented on PR #38690: URL: https://github.com/apache/doris/pull/38690#issuecomment-2262224578 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](mow) Make calculate delete bitmap timeout configurable [doris]
doris-robot commented on PR #38668: URL: https://github.com/apache/doris/pull/38668#issuecomment-2262223357 TPC-H: Total hot run time: 41456 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit aff2dd6b37b68d3b9e2bb24b5806d706bfef821d, data reload: false -- Round 1 -- q1 17689 407640594059 q2 2028201 207 201 q3 10455 124313441243 q4 10159 788 918 788 q5 7626288929482889 q6 225 140 141 140 q7 1028616 623 616 q8 9428186319681863 q9 8528660166026601 q10 8758387238303830 q11 440 255 254 254 q12 433 234 229 229 q13 17759 294129412941 q14 269 239 245 239 q15 517 489 484 484 q16 519 400 386 386 q17 958 924 921 921 q18 7869723071887188 q19 1579122612121212 q20 543 316 315 315 q21 5367477847924778 q22 352 296 279 279 Total cold run time: 112529 ms Total hot run time: 41456 ms - Round 2, with runtime_filter_mode=off - q1 4091401440254014 q2 327 220 217 217 q3 2982299331382993 q4 1976201920072007 q5 5579552554545454 q6 231 133 138 133 q7 2126180318571803 q8 3325339833493349 q9 8749866887518668 q10 3977402239413941 q11 548 453 452 452 q12 763 590 629 590 q13 16565 308831113088 q14 309 264 298 264 q15 539 496 489 489 q16 476 415 407 407 q17 1779171317091709 q18 8073773375717571 q19 1743170817231708 q20 2030185218771852 q21 5930543153965396 q22 519 445 462 445 Total cold run time: 72637 ms Total hot run time: 56550 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [FIX](regresscases) fix regress exception for test_array_element_at_and_slice [doris]
amorynan commented on PR #31621: URL: https://github.com/apache/doris/pull/31621#issuecomment-2262223948 2.0 不用合入 这是nereids 的case -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](test) Modify SQLServer image to custom hub (#38515) [doris]
hello-stephen opened a new pull request, #38690: URL: https://github.com/apache/doris/pull/38690 pick from master #38515 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](test) Modify SQLServer image to custom hub (#38515) [doris]
hello-stephen commented on PR #38690: URL: https://github.com/apache/doris/pull/38690#issuecomment-2262226246 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Json) fix some cast issue [doris]
doris-robot commented on PR #38683: URL: https://github.com/apache/doris/pull/38683#issuecomment-2262239111 TPC-DS: Total hot run time: 169501 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit b06f3e3fe1ecb4d99c8304c9fd5442f2e18607ff, data reload: false query1 905 371 367 367 query2 6452163516691635 query3 219 223 219 query4 20428 17286 17191 17191 query5 3633543 533 533 query6 299 178 163 163 query7 4601298 293 293 query8 258 202 200 200 query9 8495233323062306 query10 426 278 267 267 query11 10453 10049 98579857 query12 122 93 87 87 query13 1628382 373 373 query14 8777708679457086 query15 208 161 168 161 query16 6756462 435 435 query17 971 583 550 550 query18 1880291 297 291 query19 195 144 148 144 query20 96 85 84 84 query21 205 105 99 99 query22 4229392739913927 query23 33953 33915 33557 33557 query24 10086 311431213114 query25 689 453 443 443 query26 1691168 158 158 query27 2986285 290 285 query28 7413203120052005 query29 1196476 459 459 query30 239 158 156 156 query31 941 787 771 771 query32 108 59 59 59 query33 690 328 342 328 query34 933 505 512 505 query35 896 756 762 756 query36 1071901 901 901 query37 180 86 86 86 query38 2959279228512792 query39 892 821 835 821 query40 258 122 126 122 query41 48 46 49 46 query42 122 104 98 98 query43 482 410 416 410 query44 1208728 718 718 query45 208 185 183 183 query46 1105793 797 793 query47 1772168217031682 query48 368 301 294 294 query49 913 438 434 434 query50 910 434 442 434 query51 6790671466846684 query52 104 92 95 92 query53 256 184 190 184 query54 666 460 463 460 query55 76 80 77 77 query56 295 269 270 269 query57 1141103511501035 query58 281 264 281 264 query59 2525239423032303 query60 292 281 286 281 query61 103 94 95 94 query62 908 683 674 674 query63 213 183 176 176 query64 5651191819301918 query65 3151309131573091 query66 1326342 348 342 query67 15146 14718 15130 14718 query68 6195589 589 589 query69 763 391 348 348 query70 1116101710591017 query71 533 283 290 283 query72 8034268324882488 query73 995 328 326 326 query74 6004558455195519 query75 4388270527352705 query76 4453135213761352 query77 752 317 317 317 query78 9482891089748910 query79 2935528 533 528 query80 2048515 508 508 query81 558 219 217 217 query82 1221138 135 135 query83 269 185 174 174 query84 264 82 81 81 query85 1430310 314 310 query86 438 289 282 282 query87 3293307630933076 query88 3704239623932393 query89 411 288 281 281 query90 1974192 188 188 query91 131 101 102 101 query92 61 51 53 51 query93 3589619 611 611 query94 843 303 269 269 query95 379 268 272 268 query96 626 283 281 281 query97 3249304531223045 query98 230 202 197 197 query99 1611131113331311 Total cold run time: 272903 ms Total hot run time: 169501 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [fix](Nereids) remove db readlock before get table from db [doris]
morrySnow commented on PR #38660: URL: https://github.com/apache/doris/pull/38660#issuecomment-2262241019 run could_p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](hdfs-fs)The cache expiration should explicitly release the held fs [doris]
doris-robot commented on PR #38610: URL: https://github.com/apache/doris/pull/38610#issuecomment-2262241750 TPC-DS: Total hot run time: 170308 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit d9e3536ad4eebdd65bf77a485a0e78f21fdf8d6e, data reload: false query1 904 378 366 366 query2 6467167716431643 query3 6665214 229 214 query4 20251 17564 17234 17234 query5 3647498 504 498 query6 284 158 157 157 query7 4583300 285 285 query8 248 192 189 189 query9 8520233023152315 query10 429 290 270 270 query11 10655 10051 10069 10051 query12 121 98 88 88 query13 1645377 371 371 query14 8859800981838009 query15 205 162 170 162 query16 6776440 458 440 query17 965 569 545 545 query18 1882310 286 286 query19 196 147 151 147 query20 91 88 87 87 query21 211 104 102 102 query22 4428424740554055 query23 33665 33610 33434 33434 query24 9311308632203086 query25 662 385 413 385 query26 1438150 154 150 query27 3007285 277 277 query28 7546201019991999 query29 1073441 430 430 query30 237 156 154 154 query31 939 752 755 752 query32 102 56 58 56 query33 677 298 314 298 query34 925 492 553 492 query35 851 780 770 770 query36 1042886 874 874 query37 172 81 81 81 query38 3058283928682839 query39 919 813 828 813 query40 252 111 109 109 query41 47 43 47 43 query42 115 97 96 96 query43 484 399 420 399 query44 1165722 716 716 query45 206 183 179 179 query46 1098783 771 771 query47 1795170717421707 query48 355 286 285 285 query49 871 411 405 405 query50 911 422 433 422 query51 6844661766056605 query52 99 91 87 87 query53 250 175 175 175 query54 620 449 432 432 query55 77 74 75 74 query56 278 256 247 247 query57 1126105010351035 query58 267 265 278 265 query59 25052362 query60 324 275 274 274 query61 97 94 95 94 query62 865 679 665 665 query63 215 181 185 181 query64 5589192218671867 query65 3150308130793079 query66 1003322 336 322 query67 15256 14898 14889 14889 query68 5761568 574 568 query69 705 395 301 301 query70 1134104910641049 query71 465 272 266 266 query72 7682269125182518 query73 914 326 333 326 query74 6045567857565678 query75 3585274527332733 query76 3292138614151386 query77 572 307 302 302 query78 9461890890508908 query79 2146535 535 535 query80 2074496 513 496 query81 567 227 218 218 query82 821 131 127 127 query83 291 171 166 166 query84 268 80 76 76 query85 1442315 303 303 query86 470 317 297 297 query87 3235306731333067 query88 3715236923822369 query89 384 287 282 282 query90 1767189 187 187 query91 121 97 102 97 query92 59 49 48 48 query93 2379616 612 612 query94 767 285 295 285 query95 390 286 263 263 query96 607 284 278 278 query97 3244304430823044 query98 227 201 199 199 query99 1702127512761275 Total cold run time: 265812 ms Total hot run time: 170308 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [Improvement](mow) Make calculate delete bitmap timeout configurable [doris]
doris-robot commented on PR #38668: URL: https://github.com/apache/doris/pull/38668#issuecomment-2262242029 TPC-DS: Total hot run time: 170667 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit aff2dd6b37b68d3b9e2bb24b5806d706bfef821d, data reload: false query1 914 371 371 371 query2 6479167816421642 query3 6673218 222 218 query4 19745 17496 17399 17399 query5 3667518 539 518 query6 278 180 174 174 query7 4589307 312 307 query8 252 197 208 197 query9 8507236523252325 query10 429 284 264 264 query11 10424 995599309930 query12 117 88 89 88 query13 1619379 368 368 query14 9649790485587904 query15 196 159 165 159 query16 6816441 457 441 query17 935 559 539 539 query18 1897271 279 271 query19 183 143 143 143 query20 93 85 85 85 query21 199 104 106 104 query22 4130400538873887 query23 33951 33707 33260 33260 query24 9117316331053105 query25 661 392 433 392 query26 1402165 156 156 query27 2988282 296 282 query28 7434202320152015 query29 1015441 446 441 query30 236 160 164 160 query31 954 787 800 787 query32 107 56 59 56 query33 687 334 330 330 query34 906 510 515 510 query35 887 757 778 757 query36 1088873 913 873 query37 168 82 83 82 query38 2997281328422813 query39 933 817 808 808 query40 251 111 115 111 query41 47 42 45 42 query42 115 108 102 102 query43 474 418 430 418 query44 1207723 736 723 query45 203 180 180 180 query46 1120831 798 798 query47 1805169817171698 query48 359 285 292 285 query49 854 418 432 418 query50 907 440 439 439 query51 6765667666196619 query52 99 91 89 89 query53 261 186 185 185 query54 603 466 447 447 query55 78 79 75 75 query56 281 261 247 247 query57 1145103010121012 query58 262 265 294 265 query59 2633249223152315 query60 289 279 280 279 query61 96 90 92 90 query62 883 660 666 660 query63 213 195 187 187 query64 4858203620092009 query65 3133312931013101 query66 1257346 340 340 query67 15310 15060 14950 14950 query68 5341600 610 600 query69 776 420 319 319 query70 1132107810351035 query71 430 283 279 279 query72 7730281126072607 query73 922 329 335 329 query74 6010562956495629 query75 3428275727622757 query76 3068141414291414 query77 571 320 314 314 query78 9553892888528852 query79 2272552 543 543 query80 987 508 516 508 query81 548 233 220 220 query82 1430127 131 127 query83 201 171 174 171 query84 258 79 80 79 query85 1413323 297 297 query86 514 289 289 289 query87 3297306930973069 query88 3706240624132406 query89 397 295 291 291 query90 1759187 193 187 query91 127 99 101 99 query92 58 48 50 48 query93 2090626 621 621 query94 780 279 296 279 query95 378 267 266 266 query96 614 283 278 278 query97 3207303030703030 query98 223 202 206 202 query99 1645127712861277 Total cold run time: 263695 ms Total hot run time: 170667 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
[PR] [fix](parquet) disable parquet page index by default [doris]
morningman opened a new pull request, #38691: URL: https://github.com/apache/doris/pull/38691 there are some issue with page index parsing. disable it temporarily -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](parquet) disable parquet page index by default [doris]
doris-robot commented on PR #38691: URL: https://github.com/apache/doris/pull/38691#issuecomment-2262244720 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](jsonb)fix jsonb deserialze [doris]
amorynan commented on PR #38629: URL: https://github.com/apache/doris/pull/38629#issuecomment-2262247297 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](syntax) multi statements multi delim with semicolon [doris]
morrySnow commented on PR #38670: URL: https://github.com/apache/doris/pull/38670#issuecomment-2262247622 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Json) fix some cast issue [doris]
doris-robot commented on PR #38683: URL: https://github.com/apache/doris/pull/38683#issuecomment-2262248651 ClickBench: Total hot run time: 30.2 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit b06f3e3fe1ecb4d99c8304c9fd5442f2e18607ff, data reload: false query1 0.040.040.04 query2 0.070.040.03 query3 0.220.050.05 query4 1.690.070.07 query5 0.470.480.48 query6 1.140.720.71 query7 0.020.020.01 query8 0.050.050.04 query9 0.570.510.50 query10 0.580.560.57 query11 0.160.120.11 query12 0.140.120.12 query13 0.600.590.60 query14 0.770.800.80 query15 0.900.860.85 query16 0.350.360.37 query17 1.010.970.98 query18 0.210.210.21 query19 1.881.721.73 query20 0.010.010.01 query21 15.46 0.790.67 query22 4.347.061.54 query23 17.83 1.381.31 query24 2.270.230.21 query25 0.180.080.08 query26 0.320.210.21 query27 0.450.230.23 query28 13.16 1.010.96 query29 12.55 3.363.33 query30 0.250.060.06 query31 2.870.400.41 query32 3.240.490.48 query33 2.932.922.95 query34 15.44 4.254.24 query35 4.264.274.32 query36 0.680.470.48 query37 0.200.170.16 query38 0.160.140.16 query39 0.040.040.03 query40 0.150.130.14 query41 0.100.040.04 query42 0.050.040.05 query43 0.050.050.05 Total cold run time: 107.86 s Total hot run time: 30.2 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](parquet) disable parquet page index by default [doris]
github-actions[bot] commented on PR #38691: URL: https://github.com/apache/doris/pull/38691#issuecomment-2262249855 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](mow) Make calculate delete bitmap timeout configurable [doris]
doris-robot commented on PR #38668: URL: https://github.com/apache/doris/pull/38668#issuecomment-2262251550 ClickBench: Total hot run time: 30 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit aff2dd6b37b68d3b9e2bb24b5806d706bfef821d, data reload: false query1 0.040.040.03 query2 0.070.040.04 query3 0.220.050.05 query4 1.690.070.06 query5 0.480.490.49 query6 1.170.710.71 query7 0.030.020.01 query8 0.050.040.04 query9 0.570.520.51 query10 0.580.560.56 query11 0.160.120.12 query12 0.150.130.13 query13 0.610.600.60 query14 0.770.790.81 query15 0.880.860.88 query16 0.350.360.35 query17 0.990.980.99 query18 0.220.210.20 query19 1.831.741.72 query20 0.010.010.01 query21 15.40 0.770.65 query22 4.127.481.32 query23 17.98 1.311.24 query24 2.260.220.22 query25 0.180.070.08 query26 0.320.220.22 query27 0.460.230.22 query28 13.16 0.980.98 query29 12.52 3.313.34 query30 0.270.060.05 query31 2.860.420.40 query32 3.770.500.49 query33 2.992.962.92 query34 15.45 4.264.25 query35 4.314.324.31 query36 0.670.470.48 query37 0.210.160.16 query38 0.170.160.16 query39 0.040.040.04 query40 0.160.130.12 query41 0.100.050.05 query42 0.060.050.05 query43 0.050.040.04 Total cold run time: 108.38 s Total hot run time: 30 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](fe) Fix the default value of ReplacePartitionClause.isStrictRange [doris]
doris-robot commented on PR #38688: URL: https://github.com/apache/doris/pull/38688#issuecomment-2262249769 TPC-H: Total hot run time: 41905 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 6bc3da05d1601eb79b9f899867eeca81e7238910, data reload: false -- Round 1 -- q1 18889 427241314131 q2 2021210 208 208 q3 10444 132313131313 q4 10171 875 912 875 q5 7673302229922992 q6 226 142 142 142 q7 1052618 615 615 q8 9447194519891945 q9 8514660166126601 q10 8717380838753808 q11 427 257 260 257 q12 410 242 234 234 q13 17760 294129462941 q14 272 247 239 239 q15 538 493 486 486 q16 493 392 388 388 q17 966 935 947 935 q18 7968743272277227 q19 1380121912211219 q20 564 329 338 329 q21 5328478847384738 q22 363 286 282 282 Total cold run time: 113623 ms Total hot run time: 41905 ms - Round 2, with runtime_filter_mode=off - q1 4064404240364036 q2 329 230 220 220 q3 2994299329952993 q4 1892188918651865 q5 5292526252505250 q6 221 133 134 133 q7 2065167516821675 q8 3248329332893289 q9 8349832783138313 q10 3753386638103810 q11 547 457 443 443 q12 720 567 557 557 q13 17013 296229562956 q14 289 256 253 253 q15 517 485 481 481 q16 433 402 394 394 q17 1718169716941694 q18 7790732971827182 q19 1675168716751675 q20 1981177217601760 q21 5445523751895189 q22 511 477 473 473 Total cold run time: 70846 ms Total hot run time: 54641 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](hdfs-fs)The cache expiration should explicitly release the held fs [doris]
doris-robot commented on PR #38610: URL: https://github.com/apache/doris/pull/38610#issuecomment-2262251271 ClickBench: Total hot run time: 29.81 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit d9e3536ad4eebdd65bf77a485a0e78f21fdf8d6e, data reload: false query1 0.040.040.03 query2 0.070.040.04 query3 0.230.050.05 query4 1.670.060.07 query5 0.480.470.48 query6 1.130.710.71 query7 0.020.010.01 query8 0.060.040.04 query9 0.570.510.51 query10 0.560.590.55 query11 0.160.110.11 query12 0.150.120.13 query13 0.610.600.60 query14 0.770.800.79 query15 0.890.850.85 query16 0.360.360.35 query17 1.031.001.01 query18 0.220.210.21 query19 1.811.761.75 query20 0.020.020.01 query21 15.39 0.770.65 query22 3.888.591.12 query23 18.02 1.321.28 query24 2.270.220.21 query25 0.180.090.08 query26 0.310.210.22 query27 0.460.230.22 query28 13.16 0.990.96 query29 12.55 3.323.32 query30 0.250.060.05 query31 2.870.410.39 query32 3.260.480.48 query33 2.922.962.97 query34 15.45 4.264.27 query35 4.304.304.29 query36 0.680.480.49 query37 0.180.160.16 query38 0.160.150.15 query39 0.040.030.03 query40 0.160.130.13 query41 0.090.050.04 query42 0.050.040.04 query43 0.050.040.04 Total cold run time: 107.53 s Total hot run time: 29.81 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](parquet) disable parquet page index by default [doris]
github-actions[bot] commented on PR #38691: URL: https://github.com/apache/doris/pull/38691#issuecomment-2262255831 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](session) fix select * from variables system table [doris]
github-actions[bot] commented on PR #34529: URL: https://github.com/apache/doris/pull/34529#issuecomment-2262257070 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](session) fix select * from variables system table [doris]
github-actions[bot] commented on PR #34529: URL: https://github.com/apache/doris/pull/34529#issuecomment-2262257113 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](iceberg)revert count(*) directly returned by fe [doris]
doris-robot commented on PR #38566: URL: https://github.com/apache/doris/pull/38566#issuecomment-2262262774 TPC-H: Total hot run time: 42008 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 3e7cf484bc9cc7925d70aee8591b20167fe3, data reload: false -- Round 1 -- q1 17637 410340954095 q2 2026201 207 201 q3 10448 129313591293 q4 10182 826 916 826 q5 7676302429702970 q6 223 138 136 136 q7 1044621 626 621 q8 9460193419651934 q9 8514665266626652 q10 8773387338693869 q11 438 256 264 256 q12 447 231 233 231 q13 17755 294129432941 q14 277 245 246 245 q15 521 491 504 491 q16 526 387 385 385 q17 974 940 920 920 q18 7968731773147314 q19 1390123412211221 q20 558 321 327 321 q21 5385482348024802 q22 352 284 294 284 Total cold run time: 112574 ms Total hot run time: 42008 ms - Round 2, with runtime_filter_mode=off - q1 4042404540394039 q2 333 225 220 220 q3 3006300031443000 q4 2017206219881988 q5 5635549154485448 q6 220 132 132 132 q7 2129177918141779 q8 3328341633853385 q9 8785866387788663 q10 4000407739983998 q11 555 453 460 453 q12 788 596 617 596 q13 14740 316631513151 q14 302 272 286 272 q15 533 483 478 478 q16 469 433 424 424 q17 1778175417391739 q18 8334769077657690 q19 1734172117561721 q20 2068183018831830 q21 5810553252765276 q22 561 470 457 457 Total cold run time: 71167 ms Total hot run time: 56739 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](recycler) Support recycling data stored in different version path formats across different types of storage vaults [doris]
platoneko commented on PR #38415: URL: https://github.com/apache/doris/pull/38415#issuecomment-2262263854 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](fe) Fix the default value of ReplacePartitionClause.isStrictRange [doris]
doris-robot commented on PR #38688: URL: https://github.com/apache/doris/pull/38688#issuecomment-2262268663 TPC-DS: Total hot run time: 169238 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 6bc3da05d1601eb79b9f899867eeca81e7238910, data reload: false query1 909 375 366 366 query2 6466171616341634 query3 6679222 232 222 query4 20241 17413 17577 17413 query5 4305517 529 517 query6 283 175 164 164 query7 4605302 300 300 query8 259 194 196 194 query9 8530237423532353 query10 465 289 269 269 query11 10522 10021 10174 10021 query12 147 87 89 87 query13 1635376 377 376 query14 9574797572287228 query15 227 165 170 165 query16 7117471 494 471 query17 954 574 557 557 query18 1906298 282 282 query19 213 148 152 148 query20 104 89 87 87 query21 203 102 100 100 query22 4242392838813881 query23 33757 32802 32834 32802 query24 10349 312430313031 query25 706 404 413 404 query26 1789154 153 153 query27 2994284 286 284 query28 6979197719621962 query29 1366445 433 433 query30 285 153 152 152 query31 937 775 759 759 query32 104 57 59 57 query33 711 323 333 323 query34 899 488 487 487 query35 844 746 732 732 query36 1003875 888 875 query37 302 82 81 81 query38 2889275627672756 query39 870 805 814 805 query40 297 129 118 118 query41 48 47 50 47 query42 120 102 103 102 query43 452 411 425 411 query44 1195731 727 727 query45 209 178 180 178 query46 1102825 792 792 query47 1798170516901690 query48 367 302 294 294 query49 1192456 454 454 query50 905 444 445 444 query51 6855665266416641 query52 101 96 93 93 query53 259 187 185 185 query54 646 463 460 460 query55 80 84 81 81 query56 304 276 263 263 query57 1167105510301030 query58 309 282 275 275 query59 2694241225932412 query60 311 280 316 280 query61 119 113 114 113 query62 1061670 649 649 query63 220 189 193 189 query64 5823192518791879 query65 3271311230923092 query66 1429337 332 332 query67 15317 14763 14652 14652 query68 6888589 593 589 query69 742 371 304 304 query70 1096104711241047 query71 574 282 286 282 query72 8020268025022502 query73 981 329 337 329 query74 6006565956765659 query75 4498275527062706 query76 4939140814251408 query77 764 328 322 322 query78 9526898288308830 query79 3716542 545 542 query80 1115528 498 498 query81 576 229 230 229 query82 778 138 133 133 query83 345 182 177 177 query84 275 80 88 80 query85 1545322 305 305 query86 443 336 300 300 query87 3283311631093109 query88 3776243324222422 query89 459 289 293 289 query90 2107198 197 197 query91 129 105 101 101 query92 64 53 50 50 query93 4505616 616 616 query94 971 282 293 282 query95 391 263 275 263 query96 623 283 280 280 query97 3235306431083064 query98 210 203 195 195 query99 1631131613141314 Total cold run time: 278232 ms Total hot run time: 169238 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [I] [Bug] RoutineLoad重复消费 [doris]
dataroaring commented on issue #38551: URL: https://github.com/apache/doris/issues/38551#issuecomment-2262273360 You can set offsets when creating a new job, it consums from the specified offsets. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Revert "[feat](Nereids): Optimize Query Plan by Pulling Up Join with Common Child from Union (#36397)" [doris]
morrySnow merged PR #38561: URL: https://github.com/apache/doris/pull/38561 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (c03b564d6a1 -> 2e1219a7f56)
This is an automated email from the ASF dual-hosted git repository. morrysnow pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from c03b564d6a1 [fix](merge-on-write) segcompaction should process delete bitmap if necessary (#38369) add 2e1219a7f56 Revert "[feat](Nereids): Optimize Query Plan by Pulling Up Join with Common Child from Union (#36397)" (#38561) No new revisions were added by this update. Summary of changes: .../doris/nereids/jobs/executor/Rewriter.java | 4 +- .../org/apache/doris/nereids/rules/RuleType.java | 1 - .../nereids/rules/rewrite/PullUpJoinFromUnion.java | 484 - .../rules/rewrite/PullUpJoinFromUnionTest.java | 171 .../data/nereids_hint_tpcds_p0/shape/query14.out | 61 +-- .../data/nereids_hint_tpcds_p0/shape/query71.out | 41 +- .../data/nereids_hint_tpcds_p0/shape/query76.out | 54 ++- .../join_pull_up_union/join_pull_up_union.out | 118 - .../shape/query14.out | 61 +-- .../shape/query71.out | 45 +- .../shape/query76.out | 54 ++- .../noStatsRfPrune/query14.out | 49 ++- .../noStatsRfPrune/query71.out | 41 +- .../noStatsRfPrune/query76.out | 51 ++- .../no_stats_shape/query14.out | 63 +-- .../no_stats_shape/query71.out | 41 +- .../no_stats_shape/query76.out | 51 ++- .../rf_prune/query14.out | 49 ++- .../rf_prune/query71.out | 45 +- .../rf_prune/query76.out | 63 +-- .../nereids_tpcds_shape_sf100_p0/shape/query14.out | 61 +-- .../nereids_tpcds_shape_sf100_p0/shape/query71.out | 45 +- .../nereids_tpcds_shape_sf100_p0/shape/query76.out | 63 +-- .../tpcds_sf100/noStatsRfPrune/query14.out | 53 ++- .../tpcds_sf100/noStatsRfPrune/query71.out | 41 +- .../tpcds_sf100/noStatsRfPrune/query76.out | 63 +-- .../tpcds_sf100/no_stats_shape/query14.out | 59 +-- .../tpcds_sf100/no_stats_shape/query71.out | 41 +- .../tpcds_sf100/no_stats_shape/query76.out | 63 +-- .../new_shapes_p0/tpcds_sf100/rf_prune/query14.out | 49 ++- .../new_shapes_p0/tpcds_sf100/rf_prune/query71.out | 45 +- .../new_shapes_p0/tpcds_sf100/rf_prune/query76.out | 63 +-- .../new_shapes_p0/tpcds_sf100/shape/query14.out| 55 ++- .../new_shapes_p0/tpcds_sf100/shape/query71.out| 45 +- .../new_shapes_p0/tpcds_sf100/shape/query76.out| 63 +-- .../new_shapes_p0/tpcds_sf1000/shape/query14.out | 61 +-- .../new_shapes_p0/tpcds_sf1000/shape/query71.out | 45 +- .../new_shapes_p0/tpcds_sf1000/shape/query76.out | 63 +-- .../join_pull_up_union/join_pull_up_union.groovy | 194 - 39 files changed, 1011 insertions(+), 1708 deletions(-) delete mode 100644 fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/PullUpJoinFromUnion.java delete mode 100644 fe/fe-core/src/test/java/org/apache/doris/nereids/rules/rewrite/PullUpJoinFromUnionTest.java delete mode 100644 regression-test/data/nereids_rules_p0/join_pull_up_union/join_pull_up_union.out delete mode 100644 regression-test/suites/nereids_rules_p0/join_pull_up_union/join_pull_up_union.groovy - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](asan core) should use weak ptr in rpc context to release rf context during query context deconstructed [doris]
yiguolei merged PR #38653: URL: https://github.com/apache/doris/pull/38653 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [bugfix](asan core) should use weak ptr in rpc context to release rf context during query context deconstructed (#38653)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 7e8415fe5bb [bugfix](asan core) should use weak ptr in rpc context to
release rf context during query context deconstructed (#38653)
7e8415fe5bb is described below
commit 7e8415fe5bb0521fa3511d80a6a20cd75a1ecda6
Author: yiguolei <676222...@qq.com>
AuthorDate: Thu Aug 1 15:45:36 2024 +0800
[bugfix](asan core) should use weak ptr in rpc context to release rf
context during query context deconstructed (#38653)
## Proposed changes
Issue Number: close #xxx
F20240731 10:38:53.012670 20850 mem_tracker_limiter.cpp:135] mem tracker
label: Query#Id=7539c7d0087442b7-a1cda6392053548a, consumption: 52, peak
consumption: 1155332, mem tracker not equal to 0 when mem tracker
destruct, this usually means that memory tracking is inaccurate and
SCOPED_ATTACH_TASK and SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER are not
used correctly. 1. For query and load, memory leaks may have occurred,
it is expected that the query mem tracker will be bound to the thread
context using SCOPED_ATTACH_TASK and
SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER before all memory alloc and
free. 2. If a memory alloc is recorded by this tracker, it is expected
that be recorded in this tracker when memory is freed. 3. Merge the
remaining memory tracking value by this tracker into Orphan, if you
observe that Orphan is not equal to 0 in the mem tracker web or log,
this indicates that there may be a memory leak. 4. If you need to
transfer memory tracking value between two trackers, can use
transfer_to..[Address Sanitizer]:
0x606002867d80, size 52, strack trace:
0# Allocator::alloc_impl(unsigned long, unsigned long)
1# void* phmap::priv::Allocate<4ul, doris::vectorized::Allocator_
>(doris::vectorized::Allocator_*, unsigned long)
2# phmap::priv::raw_hash_set, phmap::Hash, phmap::EqualTo,
doris::vectorized::Allocator_ >::initialize_slots(unsigned long)
3# phmap::priv::raw_hash_set, phmap::Hash, phmap::EqualTo,
doris::vectorized::Allocator_ >::resize(unsigned long)
4# phmap::priv::raw_hash_set, phmap::Hash, phmap::EqualTo,
doris::vectorized::Allocator_ >::prepare_insert(unsigned long)
5# std::pair phmap::priv::raw_hash_set, phmap::Hash, phmap::EqualTo,
doris::vectorized::Allocator_ >::find_or_prepare_insert(int const&,
unsigned long)
6# std::pair, phmap::Hash, phmap::EqualTo, doris::vectorized::Allocator_
>::iterator, bool> phmap::priv::raw_hash_set, phmap::Hash,
phmap::EqualTo, doris::vectorized::Allocator_
>::emplace_decomposable(int const&, unsigned long, int const&)
7# std::pair, phmap::Hash, phmap::EqualTo, doris::vectorized::Allocator_
>::iterator, bool> phmap::priv::raw_hash_set, phmap::Hash,
phmap::EqualTo, doris::vectorized::Allocator_
>::EmplaceDecomposable::operator()(int const&, int const&) const
8# doris::HybridSet<(doris::PrimitiveType)5, doris::DynamicContainer,
doris::vectorized::ColumnVector >::insert(void const*)
9# doris::HybridSetBase::insert(doris::HybridSetBase*)
10# doris::RuntimePredicateWrapper::merge(doris::RuntimePredicateWrapper
const*)
11# doris::IRuntimeFilter::merge_from(doris::RuntimePredicateWrapper
const*)
12#
doris::RuntimeFilterMergeControllerEntity::merge(doris::PMergeFilterRequest
const*, butil::IOBufAsZeroCopyInputStream*)
13# doris::FragmentMgr::merge_filter(doris::PMergeFilterRequest const*,
butil::IOBufAsZeroCopyInputStream*)
14# std::_Function_handler::_M_invoke(std::_Any_data const&)
15# doris::WorkThreadPool::work_thread(int)
16# execute_native_thread_routine
17# ?
18# ?
Co-authored-by: yiguolei
---
be/src/exprs/runtime_filter.cpp | 13 +++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/be/src/exprs/runtime_filter.cpp b/be/src/exprs/runtime_filter.cpp
index 901af0472f0..d7fc2ff7490 100644
--- a/be/src/exprs/runtime_filter.cpp
+++ b/be/src/exprs/runtime_filter.cpp
@@ -1005,7 +1005,10 @@ Status IRuntimeFilter::publish(bool publish_local) {
class SyncSizeClosure : public AutoReleaseClosure> {
std::shared_ptr _dependency;
-RuntimeFilterContextSPtr _rf_context;
+// Should use weak ptr here, because when query context deconstructs,
should also delete runtime filter
+// context, it not the memory is not released. And rpc is in another
thread, it will hold rf context
+// after query context because the rpc is not returned.
+std::weak_ptr _rf_context;
std::string _rf_debug_info;
using Base =
AutoReleaseClosure>;
@@ -1021,7 +1024,13 @@ class SyncSizeClosure : public
AutoReleaseClosuresub();
if (status.is()) {
// rf merger backend may fini
Re: [PR] [fix](fe) Fix the default value of ReplacePartitionClause.isStrictRange [doris]
doris-robot commented on PR #38688: URL: https://github.com/apache/doris/pull/38688#issuecomment-2262278448 ClickBench: Total hot run time: 30.03 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 6bc3da05d1601eb79b9f899867eeca81e7238910, data reload: false query1 0.040.040.03 query2 0.080.040.04 query3 0.230.060.05 query4 1.660.080.07 query5 0.490.480.48 query6 1.120.720.71 query7 0.020.010.02 query8 0.060.050.04 query9 0.570.500.52 query10 0.570.570.56 query11 0.160.120.12 query12 0.140.130.13 query13 0.620.590.61 query14 0.780.800.80 query15 0.910.870.87 query16 0.370.360.35 query17 1.001.011.00 query18 0.220.210.21 query19 1.841.791.78 query20 0.010.010.01 query21 15.42 0.750.65 query22 4.027.511.14 query23 17.81 1.451.32 query24 2.290.220.23 query25 0.190.090.08 query26 0.330.220.22 query27 0.460.230.23 query28 13.15 1.000.97 query29 12.50 3.363.36 query30 0.260.060.06 query31 2.840.410.41 query32 3.230.500.48 query33 2.962.982.96 query34 15.49 4.264.24 query35 4.314.284.28 query36 0.680.480.47 query37 0.190.170.16 query38 0.170.150.14 query39 0.050.040.03 query40 0.160.120.12 query41 0.100.050.06 query42 0.060.050.05 query43 0.050.040.04 Total cold run time: 107.61 s Total hot run time: 30.03 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](iceberg)revert count(*) directly returned by fe [doris]
doris-robot commented on PR #38566: URL: https://github.com/apache/doris/pull/38566#issuecomment-2262281926 TPC-DS: Total hot run time: 170524 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 3e7cf484bc9cc7925d70aee8591b20167fe3, data reload: false query1 916 384 372 372 query2 6470169216921692 query3 6658236 227 227 query4 19756 17363 17434 17363 query5 3633552 539 539 query6 300 185 179 179 query7 4610318 310 310 query8 256 199 202 199 query9 8545238223572357 query10 416 292 283 283 query11 10471 10169 99659965 query12 119 91 89 89 query13 1636370 375 370 query14 10465 839077927792 query15 198 165 164 164 query16 6945505 471 471 query17 932 563 562 562 query18 1943297 293 293 query19 199 157 153 153 query20 96 91 89 89 query21 206 102 107 102 query22 4360394839553948 query23 34154 33912 33382 33382 query24 10379 312330533053 query25 684 461 405 405 query26 1729159 156 156 query27 3095287 293 287 query28 7431201319981998 query29 1255451 436 436 query30 239 172 159 159 query31 966 803 798 798 query32 110 56 60 56 query33 701 332 344 332 query34 935 498 529 498 query35 875 799 754 754 query36 1047849 873 849 query37 287 79 82 79 query38 2899276427472747 query39 882 836 831 831 query40 265 117 117 117 query41 47 42 49 42 query42 126 112 107 107 query43 487 402 423 402 query44 1202732 740 732 query45 215 185 184 184 query46 1103830 798 798 query47 1800168216941682 query48 370 295 293 293 query49 948 444 431 431 query50 915 440 440 440 query51 6772664766006600 query52 107 89 91 89 query53 265 183 181 181 query54 649 467 448 448 query55 78 77 76 76 query56 280 260 267 260 query57 1125102310351023 query58 277 326 284 284 query59 2556222624212226 query60 297 283 280 280 query61 100 94 92 92 query62 886 680 655 655 query63 216 189 194 189 query64 5631189418881888 query65 3184310630903090 query66 1319337 330 330 query67 15419 14754 14693 14693 query68 4293582 588 582 query69 757 445 320 320 query70 1101104010571040 query71 447 286 282 282 query72 7488268325412541 query73 803 341 335 335 query74 6052572456655665 query75 3422276227402740 query76 2842136214671362 query77 577 315 319 315 query78 9363893989348934 query79 1709536 541 536 query80 959 525 527 525 query81 552 225 227 225 query82 843 136 139 136 query83 255 177 176 176 query84 263 83 81 81 query85 1089312 304 304 query86 393 321 297 297 query87 3280311431643114 query88 2963248824142414 query89 384 299 292 292 query90 1699199 193 193 query91 134 117 103 103 query92 60 51 53 51 query93 1340618 622 618 query94 764 295 267 267 query95 384 273 277 273 query96 606 280 278 278 query97 3225304130393039 query98 216 205 204 204 query99 1610131113121311 Total cold run time: 263310 ms Total hot run time: 170524 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
[PR] [feature](StorageVault) Support setting path format for storage vault [doris]
platoneko opened a new pull request, #38692: URL: https://github.com/apache/doris/pull/38692 ## Proposed changes Support setting path format for storage vault -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](StorageVault) Support setting path format for storage vault [doris]
doris-robot commented on PR #38692: URL: https://github.com/apache/doris/pull/38692#issuecomment-2262282739 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](syntax) multi statements must delim with semicolon [doris]
doris-robot commented on PR #38670: URL: https://github.com/apache/doris/pull/38670#issuecomment-2262283557 TPC-H: Total hot run time: 41642 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit eb64fa02dbec7a14a5784f47fefd96e4c3134e47, data reload: false -- Round 1 -- q1 18441 426241684168 q2 2486206 204 204 q3 12293 132713791327 q4 10747 837 957 837 q5 7773303729762976 q6 222 135 135 135 q7 1040639 609 609 q8 9443177219511772 q9 8469660266346602 q10 8708383438553834 q11 431 252 246 246 q12 410 226 235 226 q13 17764 294629532946 q14 275 244 245 244 q15 523 484 502 484 q16 496 410 391 391 q17 976 898 873 873 q18 8164737772417241 q19 1391121611991199 q20 564 337 338 337 q21 5291474247104710 q22 353 281 289 281 Total cold run time: 116260 ms Total hot run time: 41642 ms - Round 2, with runtime_filter_mode=off - q1 4115405640274027 q2 328 225 220 220 q3 2983304230083008 q4 1913186418621862 q5 5260521452175214 q6 216 133 130 130 q7 2055166817001668 q8 321033053305 q9 8344831982788278 q10 3724382238413822 q11 545 464 458 458 q12 728 564 529 529 q13 13683 295929692959 q14 290 261 253 253 q15 517 477 478 477 q16 459 414 408 408 q17 1740169816971697 q18 7745732872867286 q19 1662168216731673 q20 2007175617481748 q21 5509517151195119 q22 539 460 469 460 Total cold run time: 67572 ms Total hot run time: 54601 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Doc]Update lakehouse-overview.md [doris-website]
zhangm365 opened a new pull request, #934: URL: https://github.com/apache/doris-website/pull/934 修改湖仓一体方案中文版本的表达逻辑。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](statisitcs)Disable external catalog auto analyze by default. [doris]
github-actions[bot] commented on PR #38674: URL: https://github.com/apache/doris/pull/38674#issuecomment-2262285804 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](statisitcs)Disable external catalog auto analyze by default. [doris]
github-actions[bot] commented on PR #38674: URL: https://github.com/apache/doris/pull/38674#issuecomment-2262285868 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](Cloud) Support session variable disable_file_cache in query. [doris]
wangshuo128 commented on PR #37141: URL: https://github.com/apache/doris/pull/37141#issuecomment-2262287226 run feut: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Feature](Cloud) Support session variable disable_file_cache in query. [doris]
wangshuo128 commented on PR #37141: URL: https://github.com/apache/doris/pull/37141#issuecomment-2262287380 run feut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Fix](Export) Export delete multiple times when specify the `delete_existing_files` property [doris]
github-actions[bot] commented on PR #38400: URL: https://github.com/apache/doris/pull/38400#issuecomment-2262291269 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [bugfix](iceberg)revert count(*) directly returned by fe [doris]
doris-robot commented on PR #38566: URL: https://github.com/apache/doris/pull/38566#issuecomment-2262292220 ClickBench: Total hot run time: 29.87 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 3e7cf484bc9cc7925d70aee8591b20167fe3, data reload: false query1 0.050.040.04 query2 0.070.040.03 query3 0.230.040.05 query4 1.670.070.08 query5 0.480.480.48 query6 1.130.730.71 query7 0.020.010.02 query8 0.050.040.04 query9 0.570.520.51 query10 0.580.570.56 query11 0.160.110.12 query12 0.150.130.12 query13 0.610.600.60 query14 0.790.790.80 query15 0.900.860.85 query16 0.350.360.35 query17 1.000.970.98 query18 0.220.200.22 query19 1.811.751.77 query20 0.010.010.01 query21 15.39 0.770.66 query22 4.558.471.16 query23 17.99 1.271.28 query24 2.300.220.21 query25 0.180.080.09 query26 0.320.200.21 query27 0.460.230.23 query28 13.17 1.000.96 query29 12.53 3.363.32 query30 0.270.060.05 query31 2.870.400.40 query32 3.250.480.48 query33 2.922.992.96 query34 15.45 4.294.29 query35 4.304.294.27 query36 0.680.490.49 query37 0.190.160.17 query38 0.160.150.14 query39 0.040.030.04 query40 0.160.140.14 query41 0.090.050.04 query42 0.060.050.05 query43 0.050.040.05 Total cold run time: 108.23 s Total hot run time: 29.87 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](StorageVault) Support setting path format for storage vault [doris]
github-actions[bot] commented on PR #38692: URL: https://github.com/apache/doris/pull/38692#issuecomment-2262295242 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](syntax) multi statements must delim with semicolon [doris]
doris-robot commented on PR #38670: URL: https://github.com/apache/doris/pull/38670#issuecomment-2262303200 TPC-DS: Total hot run time: 169176 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit eb64fa02dbec7a14a5784f47fefd96e4c3134e47, data reload: false query1 923 380 369 369 query2 6491168316671667 query3 6675218 228 218 query4 20342 17481 17187 17187 query5 4335506 523 506 query6 288 168 175 168 query7 4600311 283 283 query8 261 196 197 196 query9 8540236223512351 query10 433 280 284 280 query11 10426 987610052 9876 query12 134 95 90 90 query13 1641378 383 378 query14 9415863678907890 query15 217 162 165 162 query16 7028475 418 418 query17 971 564 550 550 query18 1763296 282 282 query19 208 147 151 147 query20 92 85 87 85 query21 207 103 103 103 query22 4159408839013901 query23 33871 33009 32955 32955 query24 10332 304530113011 query25 727 404 410 404 query26 1763150 151 150 query27 2972277 288 277 query28 6897195619461946 query29 1254432 422 422 query30 291 149 151 149 query31 960 766 753 753 query32 101 57 57 57 query33 710 315 395 315 query34 913 480 500 480 query35 831 726 721 721 query36 996 838 858 838 query37 285 79 91 79 query38 2829276927352735 query39 879 800 814 800 query40 279 112 110 110 query41 47 46 43 43 query42 117 103 100 100 query43 477 404 418 404 query44 1194714 743 714 query45 207 182 176 176 query46 1082828 802 802 query47 1756169017061690 query48 368 290 289 289 query49 1193439 441 439 query50 902 432 461 432 query51 6835668966776677 query52 100 93 88 88 query53 259 186 178 178 query54 643 463 464 463 query55 78 77 78 77 query56 283 256 263 256 query57 1160102610301026 query58 277 291 263 263 query59 2700243124982431 query60 289 269 265 265 query61 102 97 92 92 query62 932 672 659 659 query63 215 185 193 185 query64 5918193018721872 query65 3151307230693069 query66 1431340 334 334 query67 15246 14745 14935 14745 query68 4471574 590 574 query69 511 288 305 288 query70 1130105110761051 query71 446 274 274 274 query72 8006267524892489 query73 766 331 328 328 query74 6038566455795579 query75 3807274427302730 query76 2710132113841321 query77 688 324 300 300 query78 9534884788588847 query79 2695537 528 528 query80 1066511 507 507 query81 555 230 227 227 query82 778 131 128 128 query83 327 171 171 171 query84 271 77 77 77 query85 1891362 299 299 query86 470 282 322 282 query87 3244313031083108 query88 3896241623892389 query89 390 298 287 287 query90 1986195 197 195 query91 126 100 100 100 query92 69 53 53 53 query93 2275617 613 613 query94 1000293 299 293 query95 380 265 268 265 query96 600 280 279 279 query97 3235301430463014 query98 216 192 200 192 query99 1651131013121310 Total cold run time: 268229 ms Total hot run time: 169176 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
(doris) branch branch-2.1 updated: [Improvement](schema scan) Use async scanner for schema scanners (#38… (#38666)
This is an automated email from the ASF dual-hosted git repository.
gabriellee pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new 9d23ccf1f2a [Improvement](schema scan) Use async scanner for schema
scanners (#38… (#38666)
9d23ccf1f2a is described below
commit 9d23ccf1f2ae3fd6c3f3adc7af428c8d116f13db
Author: Gabriel
AuthorDate: Thu Aug 1 16:05:24 2024 +0800
[Improvement](schema scan) Use async scanner for schema scanners (#38…
(#38666)
…403)
---
be/src/exec/schema_scanner.cpp | 69 +-
be/src/exec/schema_scanner.h | 24 +++-
.../schema_active_queries_scanner.cpp | 2 +-
.../schema_scanner/schema_active_queries_scanner.h | 2 +-
.../schema_scanner/schema_backend_active_tasks.cpp | 3 +-
.../schema_scanner/schema_backend_active_tasks.h | 2 +-
.../schema_scanner/schema_charsets_scanner.cpp | 2 +-
.../exec/schema_scanner/schema_charsets_scanner.h | 2 +-
.../schema_scanner/schema_collations_scanner.cpp | 2 +-
.../schema_scanner/schema_collations_scanner.h | 2 +-
.../exec/schema_scanner/schema_columns_scanner.cpp | 2 +-
.../exec/schema_scanner/schema_columns_scanner.h | 2 +-
.../exec/schema_scanner/schema_dummy_scanner.cpp | 2 +-
be/src/exec/schema_scanner/schema_dummy_scanner.h | 2 +-
.../exec/schema_scanner/schema_files_scanner.cpp | 2 +-
be/src/exec/schema_scanner/schema_files_scanner.h | 2 +-
.../schema_metadata_name_ids_scanner.cpp | 2 +-
.../schema_metadata_name_ids_scanner.h | 2 +-
.../schema_scanner/schema_partitions_scanner.cpp | 2 +-
.../schema_scanner/schema_partitions_scanner.h | 2 +-
.../schema_scanner/schema_processlist_scanner.cpp | 2 +-
.../schema_scanner/schema_processlist_scanner.h| 2 +-
.../schema_scanner/schema_profiling_scanner.cpp| 2 +-
.../exec/schema_scanner/schema_profiling_scanner.h | 2 +-
.../exec/schema_scanner/schema_routine_scanner.cpp | 2 +-
.../exec/schema_scanner/schema_routine_scanner.h | 2 +-
.../exec/schema_scanner/schema_rowsets_scanner.cpp | 2 +-
.../exec/schema_scanner/schema_rowsets_scanner.h | 2 +-
.../schema_schema_privileges_scanner.cpp | 2 +-
.../schema_schema_privileges_scanner.h | 2 +-
.../schema_scanner/schema_schemata_scanner.cpp | 2 +-
.../exec/schema_scanner/schema_schemata_scanner.h | 2 +-
.../schema_table_privileges_scanner.cpp| 2 +-
.../schema_table_privileges_scanner.h | 2 +-
.../exec/schema_scanner/schema_tables_scanner.cpp | 2 +-
be/src/exec/schema_scanner/schema_tables_scanner.h | 2 +-
.../schema_user_privileges_scanner.cpp | 2 +-
.../schema_user_privileges_scanner.h | 2 +-
be/src/exec/schema_scanner/schema_user_scanner.cpp | 2 +-
be/src/exec/schema_scanner/schema_user_scanner.h | 2 +-
.../schema_scanner/schema_variables_scanner.cpp| 2 +-
.../exec/schema_scanner/schema_variables_scanner.h | 2 +-
.../exec/schema_scanner/schema_views_scanner.cpp | 2 +-
be/src/exec/schema_scanner/schema_views_scanner.h | 2 +-
.../schema_workload_groups_scanner.cpp | 2 +-
.../schema_workload_groups_scanner.h | 2 +-
.../schema_workload_sched_policy_scanner.cpp | 3 +-
.../schema_workload_sched_policy_scanner.h | 2 +-
be/src/pipeline/exec/schema_scan_operator.cpp | 9 ++-
be/src/pipeline/exec/schema_scan_operator.h| 14 -
be/src/pipeline/pipeline_x/dependency.h| 16 +
be/src/pipeline/pipeline_x/pipeline_x_task.cpp | 2 -
be/src/vec/exec/vschema_scan_node.cpp | 2 +-
53 files changed, 162 insertions(+), 68 deletions(-)
diff --git a/be/src/exec/schema_scanner.cpp b/be/src/exec/schema_scanner.cpp
index 2b6b2c1f3c0..d9cafcf9049 100644
--- a/be/src/exec/schema_scanner.cpp
+++ b/be/src/exec/schema_scanner.cpp
@@ -50,7 +50,10 @@
#include "exec/schema_scanner/schema_workload_groups_scanner.h"
#include "exec/schema_scanner/schema_workload_sched_policy_scanner.h"
#include "olap/hll.h"
+#include "pipeline/pipeline_x/dependency.h"
#include "runtime/define_primitive_type.h"
+#include "runtime/fragment_mgr.h"
+#include "runtime/types.h"
#include "util/string_util.h"
#include "util/types.h"
#include "vec/columns/column.h"
@@ -64,6 +67,7 @@
#include "vec/core/column_with_type_and_name.h"
#include "vec/core/types.h"
#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_factory.hpp"
namespace doris {
class ObjectPool;
@@ -84,7 +88,60 @@ Status SchemaScanner::start(RuntimeState* state) {
return Status::OK();
}
-Status SchemaScanner::get_next_block(vectorized::Block* block, bool* eos) {
+Status SchemaScanner::get_next_block(RuntimeState* state, vectorized::Block*
block, bool* eos) {
+if (_data_block ==
Re: [PR] [fix](local exchange) Fix local exchange blocked by a huge data block [doris]
Gabriel39 merged PR #38657: URL: https://github.com/apache/doris/pull/38657 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](cloud) Add lazy commit mechanism for `commit_txn` [doris]
doris-robot commented on PR #38243: URL: https://github.com/apache/doris/pull/38243#issuecomment-2262300969 TPC-H: Total hot run time: 41825 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit f40d3af909e3a4ce446ee5217827a6811bce34d1, data reload: false -- Round 1 -- q1 17655 440740714071 q2 2028208 202 202 q3 10546 137913731373 q4 10237 818 935 818 q5 7634302229762976 q6 219 135 140 135 q7 1039612 606 606 q8 9443188119881881 q9 8530662966216621 q10 8797390438923892 q11 435 249 239 239 q12 412 230 225 225 q13 17774 296929452945 q14 269 250 247 247 q15 517 479 484 479 q16 518 380 393 380 q17 998 889 918 889 q18 8073728673557286 q19 1490122212201220 q20 549 341 342 341 q21 5317472447984724 q22 349 282 275 275 Total cold run time: 112829 ms Total hot run time: 41825 ms - Round 2, with runtime_filter_mode=off - q1 4136403240024002 q2 324 218 216 216 q3 2992304431853044 q4 2028203920062006 q5 5679547054735470 q6 220 130 130 130 q7 2187185918191819 q8 3341339033723372 q9 8688871187828711 q10 3962413639333933 q11 546 455 476 455 q12 781 601 580 580 q13 16313 315731323132 q14 299 277 276 276 q15 530 497 487 487 q16 470 415 440 415 q17 1755174817371737 q18 8350777977267726 q19 1740170717101707 q20 2054189418281828 q21 5857546854195419 q22 531 450 475 450 Total cold run time: 72783 ms Total hot run time: 56915 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [doc](routine load) update routine load doc [doris-website]
sollhui opened a new pull request, #935: URL: https://github.com/apache/doris-website/pull/935 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated (9d23ccf1f2a -> cafcf7acc1f)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git from 9d23ccf1f2a [Improvement](schema scan) Use async scanner for schema scanners (#38… (#38666) add cafcf7acc1f [cherry-pick](SSL) Fix ssl connection close 2.1 (#38587) (#38677) No new revisions were added by this update. Summary of changes: fe/fe-core/src/main/java/org/apache/doris/mysql/MysqlChannel.java| 2 +- .../src/main/java/org/apache/doris/qe/MysqlConnectProcessor.java | 5 + 2 files changed, 6 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](syntax) multi statements must delim with semicolon [doris]
doris-robot commented on PR #38670: URL: https://github.com/apache/doris/pull/38670#issuecomment-2262313846 ClickBench: Total hot run time: 30.29 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit eb64fa02dbec7a14a5784f47fefd96e4c3134e47, data reload: false query1 0.040.030.03 query2 0.070.040.04 query3 0.230.050.05 query4 1.680.060.06 query5 0.480.490.48 query6 1.160.720.72 query7 0.020.010.01 query8 0.050.040.04 query9 0.570.510.50 query10 0.570.550.58 query11 0.160.110.11 query12 0.140.130.13 query13 0.620.610.60 query14 0.780.800.80 query15 0.900.860.86 query16 0.350.340.35 query17 1.000.990.99 query18 0.220.210.21 query19 1.881.751.80 query20 0.010.010.01 query21 15.42 0.770.65 query22 3.847.181.48 query23 18.04 1.361.38 query24 2.320.210.22 query25 0.180.070.08 query26 0.320.220.22 query27 0.460.240.24 query28 13.16 1.000.97 query29 12.56 3.393.35 query30 0.260.060.06 query31 2.860.410.41 query32 3.230.490.48 query33 2.943.022.94 query34 15.45 4.244.26 query35 4.294.264.28 query36 0.670.480.47 query37 0.200.160.16 query38 0.160.150.15 query39 0.040.030.04 query40 0.150.120.14 query41 0.110.050.05 query42 0.060.050.05 query43 0.050.040.04 Total cold run time: 107.7 s Total hot run time: 30.29 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Bug] RoutineLoad重复消费 [doris]
CoderXionghs commented on issue #38551: URL: https://github.com/apache/doris/issues/38551#issuecomment-2262305660 > You can set offsets when creating a new job, it consums from the specified offsets. thks for your answer; my topic has many partitions(≥50),so set the offsets will be very difficult; what i hope is can routine load work like other normal kafka client，when auto.offset.reset=earliest，it consume from the last offset，if the last offset not exist, it consume from the beginning. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [doc](routine load) update routine load doc [doris-website]
sollhui commented on PR #935: URL: https://github.com/apache/doris-website/pull/935#issuecomment-2262308673 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Improvement](schema scan) Use async scanner for schema scanners (#38… [doris]
Gabriel39 merged PR #38666: URL: https://github.com/apache/doris/pull/38666 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](local exchange) Fix local exchange blocked by a huge data block (#38657)
This is an automated email from the ASF dual-hosted git repository.
gabriellee pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new bb1a9c98d42 [fix](local exchange) Fix local exchange blocked by a huge
data block (#38657)
bb1a9c98d42 is described below
commit bb1a9c98d42c224fefb1140158a7c52ec651c1c3
Author: Gabriel
AuthorDate: Thu Aug 1 16:05:53 2024 +0800
[fix](local exchange) Fix local exchange blocked by a huge data block
(#38657)
If a huge block is push into local exchanger, it will be blocked due to
concurrent problems. This PR use a unique lock to resolve it .
---
.../local_exchange/local_exchange_sink_operator.h | 2 +
.../local_exchange_source_operator.cpp | 6 +-
.../local_exchange_source_operator.h | 2 +
be/src/pipeline/local_exchange/local_exchanger.cpp | 115 +
be/src/pipeline/local_exchange/local_exchanger.h | 14 +++
5 files changed, 72 insertions(+), 67 deletions(-)
diff --git a/be/src/pipeline/local_exchange/local_exchange_sink_operator.h
b/be/src/pipeline/local_exchange/local_exchange_sink_operator.h
index 0ff1df26001..faa48d209f4 100644
--- a/be/src/pipeline/local_exchange/local_exchange_sink_operator.h
+++ b/be/src/pipeline/local_exchange/local_exchange_sink_operator.h
@@ -56,6 +56,8 @@ private:
friend class PassToOneExchanger;
friend class LocalMergeSortExchanger;
friend class AdaptivePassthroughExchanger;
+template
+friend class Exchanger;
ExchangerBase* _exchanger = nullptr;
diff --git a/be/src/pipeline/local_exchange/local_exchange_source_operator.cpp
b/be/src/pipeline/local_exchange/local_exchange_source_operator.cpp
index 56f0a157cde..6b0cca2d71a 100644
--- a/be/src/pipeline/local_exchange/local_exchange_source_operator.cpp
+++ b/be/src/pipeline/local_exchange/local_exchange_source_operator.cpp
@@ -75,11 +75,13 @@ std::string LocalExchangeSourceLocalState::debug_string(int
indentation_level) c
fmt::memory_buffer debug_string_buffer;
fmt::format_to(debug_string_buffer,
"{}, _channel_id: {}, _num_partitions: {}, _num_senders:
{}, _num_sources: {}, "
- "_running_sink_operators: {}, _running_source_operators:
{}, mem_usage: {}",
+ "_running_sink_operators: {}, _running_source_operators:
{}, mem_usage: {}, "
+ "data queue info: {}",
Base::debug_string(indentation_level), _channel_id,
_exchanger->_num_partitions,
_exchanger->_num_senders, _exchanger->_num_sources,
_exchanger->_running_sink_operators,
_exchanger->_running_source_operators,
- _shared_state->mem_usage.load());
+ _shared_state->mem_usage.load(),
+ _exchanger->data_queue_debug_string(_channel_id));
size_t i = 0;
fmt::format_to(debug_string_buffer, ", MemTrackers: ");
for (auto* mem_tracker : _shared_state->mem_trackers) {
diff --git a/be/src/pipeline/local_exchange/local_exchange_source_operator.h
b/be/src/pipeline/local_exchange/local_exchange_source_operator.h
index f9fa4cfa4ed..d2f68d4ebac 100644
--- a/be/src/pipeline/local_exchange/local_exchange_source_operator.h
+++ b/be/src/pipeline/local_exchange/local_exchange_source_operator.h
@@ -51,6 +51,8 @@ private:
friend class PassToOneExchanger;
friend class LocalMergeSortExchanger;
friend class AdaptivePassthroughExchanger;
+template
+friend class Exchanger;
ExchangerBase* _exchanger = nullptr;
int _channel_id;
diff --git a/be/src/pipeline/local_exchange/local_exchanger.cpp
b/be/src/pipeline/local_exchange/local_exchanger.cpp
index 27b7fc7e7fd..647ddcfba2d 100644
--- a/be/src/pipeline/local_exchange/local_exchanger.cpp
+++ b/be/src/pipeline/local_exchange/local_exchanger.cpp
@@ -26,6 +26,37 @@
namespace doris::pipeline {
+template
+bool Exchanger::_enqueue_data_and_set_ready(int channel_id,
+
LocalExchangeSinkLocalState& local_state,
+ BlockType&& block) {
+std::unique_lock l(_m);
+if (_data_queue[channel_id].enqueue(std::move(block))) {
+local_state._shared_state->set_ready_to_read(channel_id);
+return true;
+}
+return false;
+}
+
+template
+bool Exchanger::_dequeue_data(LocalExchangeSourceLocalState&
local_state,
+ BlockType& block, bool* eos) {
+bool all_finished = _running_sink_operators == 0;
+if (_data_queue[local_state._channel_id].try_dequeue(block)) {
+return true;
+} else if (all_finished) {
+*eos = true;
+} else {
+std::unique_lock l(_m);
+if (_data_queue[local_state._channel_id].try_dequeue(block)) {
+return true;
+}
+
Re: [PR] [feature](http action)Add http action to show nested inverted index file (#38272) [doris]
qidaye commented on PR #38672: URL: https://github.com/apache/doris/pull/38672#issuecomment-2262315753 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [cherry-pick](SSL) Fix ssl connection close 2.1 (#38587) [doris]
dataroaring merged PR #38677: URL: https://github.com/apache/doris/pull/38677 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](local exchange) Fix local exchange blocked by a huge data block… [doris]
doris-robot commented on PR #38693: URL: https://github.com/apache/doris/pull/38693#issuecomment-2262317607 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](local exchange) Fix local exchange blocked by a huge data block… [doris]
Gabriel39 opened a new pull request, #38693: URL: https://github.com/apache/doris/pull/38693 … (#38657) If a huge block is push into local exchanger, it will be blocked due to concurrent problems. This PR use a unique lock to resolve it . ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](local exchange) Fix local exchange blocked by a huge data block… [doris]
Gabriel39 commented on PR #38693: URL: https://github.com/apache/doris/pull/38693#issuecomment-2262317910 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](case) fix jsonb-load output [doris]
github-actions[bot] commented on PR #38661: URL: https://github.com/apache/doris/pull/38661#issuecomment-2262318766 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](case) fix jsonb-load output [doris]
github-actions[bot] commented on PR #38661: URL: https://github.com/apache/doris/pull/38661#issuecomment-2262318833 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](mtmv) Choose a valid partition column when there are both valid and invalid expressions (#38367) [doris]
liutang123 commented on PR #38684: URL: https://github.com/apache/doris/pull/38684#issuecomment-2262319628 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](cloud) Add lazy commit mechanism for `commit_txn` [doris]
doris-robot commented on PR #38243: URL: https://github.com/apache/doris/pull/38243#issuecomment-2262321636 TPC-DS: Total hot run time: 170685 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit f40d3af909e3a4ce446ee5217827a6811bce34d1, data reload: false query1 919 381 370 370 query2 6455170816701670 query3 6649224 221 221 query4 20265 17544 17285 17285 query5 3664517 520 517 query6 284 172 178 172 query7 4598305 305 305 query8 274 201 205 201 query9 8506236923562356 query10 434 282 278 278 query11 10519 10016 10112 10016 query12 122 88 97 88 query13 1646384 366 366 query14 8846788978577857 query15 210 199 163 163 query16 6672462 423 423 query17 943 586 542 542 query18 1888287 277 277 query19 194 142 159 142 query20 94 85 84 84 query21 204 104 99 99 query22 4261412439283928 query23 34031 33736 33739 33736 query24 10288 314631293129 query25 679 415 403 403 query26 1706154 157 154 query27 3160282 285 282 query28 7380202519971997 query29 1187441 447 441 query30 240 155 154 154 query31 955 791 808 791 query32 102 59 57 57 query33 672 328 329 328 query34 913 497 503 497 query35 855 736 773 736 query36 1054890 891 890 query37 272 83 81 81 query38 2987284927812781 query39 861 805 855 805 query40 249 118 111 111 query41 49 45 45 45 query42 130 97 100 97 query43 469 434 414 414 query44 1167712 744 712 query45 215 185 180 180 query46 1102823 768 768 query47 1816173917111711 query48 360 298 287 287 query49 924 420 403 403 query50 910 429 424 424 query51 6840680266786678 query52 95 88 88 88 query53 251 181 184 181 query54 621 456 441 441 query55 77 74 74 74 query56 273 266 258 258 query57 1124107610411041 query58 278 257 299 257 query59 2764238222812281 query60 302 261 275 261 query61 96 94 94 94 query62 887 667 677 667 query63 211 188 186 186 query64 5712190218991899 query65 3193309730803080 query66 1321331 334 331 query67 15391 14910 14715 14715 query68 5484560 575 560 query69 701 379 307 307 query70 1109109610441044 query71 468 282 272 272 query72 7551273225042504 query73 929 333 326 326 query74 6115573756975697 query75 3620273527252725 query76 3217136713951367 query77 538 301 302 301 query78 9488888388868883 query79 1686533 528 528 query80 1908507 491 491 query81 570 225 225 225 query82 478 132 127 127 query83 275 172 165 165 query84 277 77 77 77 query85 1162355 426 355 query86 448 308 309 308 query87 3321316831333133 query88 3691238423862384 query89 379 284 291 284 query90 1688186 186 186 query91 125 99 99 99 query92 65 51 50 50 query93 2016621 632 621 query94 807 286 297 286 query95 381 270 266 266 query96 595 280 276 276 query97 3207311530593059 query98 246 204 196 196 query99 1628129112771277 Total cold run time: 265989 ms Total hot run time: 170685 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the
Re: [PR] [COMPILE](macos) do not patch bitshuffle on macos [doris]
github-actions[bot] commented on PR #38634:
URL: https://github.com/apache/doris/pull/38634#issuecomment-2262324716
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/10194559858";) output.
shellcheck errors
```
'shellcheck ' returned error 1 finding the following syntactical issues:
--
In thirdparty/build-thirdparty.sh line 1169:
if [[ "${MACHINE_OS}" != "Darwin" ]] && ([[ "${arch}" == "avx2" ]]
|| [[ "${arch}" == "avx512" ]] || [[ "${arch}" == "neon" ]]); then
^-- SC2235 (style): Use {
..; } instead of (..) to avoid subshell overhead.
For more information:
https://www.shellcheck.net/wiki/SC2235 -- Use { ..; } instead of (..) to
av...
--
You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
# shellcheck disable=
above the line that contains the issue, where is the error code;
3. Add '-e ' to the SHELLCHECK_OPTS setting in your .yml action file.
```
shfmt errors
```
'shfmt ' found no issues.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [COMPILE](macos) do not patch bitshuffle on macos [doris]
Mryange commented on PR #38634: URL: https://github.com/apache/doris/pull/38634#issuecomment-2262323527 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pipeline](fix) Fix blocking task which is not triggered by 2nd RPC (… [doris]
doris-robot commented on PR #38694: URL: https://github.com/apache/doris/pull/38694#issuecomment-2262326073 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [draft](cloud-merge) Support shadow tablet to do cumulative compaction in cloud mode [doris]
Lchangliang commented on PR #37293: URL: https://github.com/apache/doris/pull/37293#issuecomment-2262326100 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [pipeline](fix) Fix blocking task which is not triggered by 2nd RPC (… [doris]
Gabriel39 commented on PR #38694: URL: https://github.com/apache/doris/pull/38694#issuecomment-2262326175 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](http action)Add http action to show nested inverted index file (#38272) [doris]
github-actions[bot] commented on code in PR #38672: URL: https://github.com/apache/doris/pull/38672#discussion_r1699632063 ## be/src/olap/rowset/beta_rowset.cpp: ## @@ -702,4 +703,131 @@ Status BetaRowset::calc_local_file_crc(uint32_t* crc_value, int64_t* file_count) return Status::OK(); } +Status BetaRowset::show_nested_index_file(rapidjson::Value* rowset_value, Review Comment: warning: function 'show_nested_index_file' exceeds recommended size/complexity thresholds [readability-function-size] ```cpp Status BetaRowset::show_nested_index_file(rapidjson::Value* rowset_value, ^ ``` Additional context **be/src/olap/rowset/beta_rowset.cpp:705:** 124 lines including whitespace and comments (threshold 80) ```cpp Status BetaRowset::show_nested_index_file(rapidjson::Value* rowset_value, ^ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [pipeline](fix) Fix blocking task which is not triggered by 2nd RPC (… [doris]
Gabriel39 opened a new pull request, #38694: URL: https://github.com/apache/doris/pull/38694 …#38568) Once a query is cancelled due to any reason, BE may not receive 2nd RPC from FE. If so, we must ensure the execution dependency is ready so tasks will not be blocked. ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](compatibility) type toSql should return lowercase string (#38012) [doris]
morrySnow commented on PR #38521: URL: https://github.com/apache/doris/pull/38521#issuecomment-2262327894 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](Nereids) remove db readlock before get table from db [doris]
morrySnow commented on PR #38660: URL: https://github.com/apache/doris/pull/38660#issuecomment-2262329132 run cloud_p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](local exchange) Fix local exchange blocked by a huge data block… [doris]
github-actions[bot] commented on PR #38693: URL: https://github.com/apache/doris/pull/38693#issuecomment-2262330631 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [feature](cloud) Add lazy commit mechanism for `commit_txn` [doris]
doris-robot commented on PR #38243: URL: https://github.com/apache/doris/pull/38243#issuecomment-2262331605 ClickBench: Total hot run time: 30.72 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit f40d3af909e3a4ce446ee5217827a6811bce34d1, data reload: false query1 0.040.030.03 query2 0.070.030.04 query3 0.230.050.04 query4 1.710.070.09 query5 0.490.490.50 query6 1.140.720.72 query7 0.020.020.02 query8 0.050.040.04 query9 0.560.520.51 query10 0.570.570.55 query11 0.160.120.12 query12 0.150.120.12 query13 0.610.600.60 query14 0.780.790.77 query15 0.900.860.87 query16 0.350.360.35 query17 1.010.980.98 query18 0.220.200.20 query19 1.871.751.73 query20 0.010.010.01 query21 15.40 0.780.67 query22 3.846.742.14 query23 18.03 1.281.21 query24 2.240.230.23 query25 0.180.080.07 query26 0.320.210.21 query27 0.460.240.23 query28 13.16 1.010.97 query29 12.53 3.323.29 query30 0.260.050.06 query31 2.850.410.40 query32 3.240.490.50 query33 2.952.943.00 query34 15.44 4.264.29 query35 4.294.304.26 query36 0.670.470.48 query37 0.190.170.16 query38 0.160.140.15 query39 0.040.040.03 query40 0.160.130.13 query41 0.100.050.05 query42 0.060.040.05 query43 0.040.050.04 Total cold run time: 107.55 s Total hot run time: 30.72 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](clone) fix stale tablet report miss the new cloning replica [doris]
yujun777 opened a new pull request, #38695: URL: https://github.com/apache/doris/pull/38695 BUG: 1. BE begin collect tablet report; 2. BE clone a new replica A; 3. FE handle this BE's tablet report from step 1. But it's stale, it don't include the replica A, then FE mark replica A as bad; only after 1min later, BE report tablets again, then the new report contains replica A, only after that, FE will change replica A from bad to good. Fix: If BE clone a new replica, it should increase its report version and tell FE to update it. Then if FE handle the stale tablet report, it will compare BE's report version, then found the tablet report is stale and discard it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](clone) fix stale tablet report miss the new cloning replica [doris]
doris-robot commented on PR #38695: URL: https://github.com/apache/doris/pull/38695#issuecomment-2262333990 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](clone) fix stale tablet report miss the new cloning replica [doris]
yujun777 commented on PR #38695: URL: https://github.com/apache/doris/pull/38695#issuecomment-2262334117 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [COMPILE](macos) do not patch bitshuffle on macos [doris]
github-actions[bot] commented on PR #38634: URL: https://github.com/apache/doris/pull/38634#issuecomment-2262334810 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [draft](cloud-merge) Support shadow tablet to do cumulative compaction in cloud mode [doris]
github-actions[bot] commented on code in PR #37293:
URL: https://github.com/apache/doris/pull/37293#discussion_r1699638552
##
cloud/test/meta_service_job_test.cpp:
##
@@ -2294,4 +2310,212 @@ TEST(MetaServiceJobTest, SchemaChangeJobPersistTest) {
ASSERT_EQ(res.status().code(), MetaServiceCode::OK);
}
+TEST(MetaServiceJobTest, DoCompactionWhenSC) {
Review Comment:
warning: function 'TEST' exceeds recommended size/complexity thresholds
[readability-function-size]
```cpp
TEST(MetaServiceJobTest, DoCompactionWhenSC) {
^
```
Additional context
**cloud/test/meta_service_job_test.cpp:2312:** 85 lines including whitespace
and comments (threshold 80)
```cpp
TEST(MetaServiceJobTest, DoCompactionWhenSC) {
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
