[GitHub] [doris-website] jeffreys-cat opened a new pull request, #60: [Actions] add secrets.GITHUB_TOKEN to env

[GitBox]

jeffreys-cat opened a new pull request, #60:
URL: https://github.com/apache/doris-website/pull/60

   - [Actions] add secrets.GITHUB_TOKEN to env


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] wsjz merged pull request #11934: [feature-wip](parquet-reader) read data by parquet scanner

[GitBox]

wsjz merged PR #11934:
URL: https://github.com/apache/doris/pull/11934


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] yiguolei merged pull request #11859: [fix](agg)repeat node shouldn't change slot's nullable property of agg node

[GitBox]

yiguolei merged PR #11859:
URL: https://github.com/apache/doris/pull/11859


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[doris] branch master updated: [fix](agg)repeat node shouldn't change slot's nullable property of agg node (#11859)

[yiguolei]

This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 0c5b4ecc7c [fix](agg)repeat node shouldn't change slot's nullable 
property of agg node (#11859)
0c5b4ecc7c is described below

commit 0c5b4ecc7cc46e2b014bb1f4c909be04f736c54a
Author: starocean999 <40539150+starocean...@users.noreply.github.com>
AuthorDate: Mon Aug 22 16:28:45 2022 +0800

[fix](agg)repeat node shouldn't change slot's nullable property of agg node 
(#11859)
---
 fe/fe-core/src/main/java/org/apache/doris/planner/RepeatNode.java | 8 
 1 file changed, 8 deletions(-)

diff --git a/fe/fe-core/src/main/java/org/apache/doris/planner/RepeatNode.java 
b/fe/fe-core/src/main/java/org/apache/doris/planner/RepeatNode.java
index 5fef371ec8..f9f136f208 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/planner/RepeatNode.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/planner/RepeatNode.java
@@ -24,10 +24,8 @@ import org.apache.doris.analysis.GroupByClause;
 import org.apache.doris.analysis.GroupingInfo;
 import org.apache.doris.analysis.SlotDescriptor;
 import org.apache.doris.analysis.SlotId;
-import org.apache.doris.analysis.SlotRef;
 import org.apache.doris.analysis.TupleDescriptor;
 import org.apache.doris.analysis.TupleId;
-import org.apache.doris.analysis.VirtualSlotRef;
 import org.apache.doris.common.UserException;
 import org.apache.doris.statistics.StatisticalType;
 import org.apache.doris.statistics.StatsRecursiveDerive;
@@ -126,12 +124,6 @@ public class RepeatNode extends PlanNode {
 
outputTupleDesc.setTable(analyzer.getTupleDesc(inputTupleIds.get(0)).getTable());
 }
 
-//set aggregate nullable
-for (Expr slot : groupByClause.getGroupingExprs()) {
-if (slot instanceof SlotRef && !(slot instanceof VirtualSlotRef)) {
-((SlotRef) slot).getDesc().setIsNullable(true);
-}
-}
 outputTupleDesc.computeStatAndMemLayout();
 
 List> groupingIdList = new ArrayList<>();


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris-website] jeffreys-cat closed pull request #60: [Actions] add secrets.GITHUB_TOKEN to env

[GitBox]

jeffreys-cat closed pull request #60: [Actions] add secrets.GITHUB_TOKEN to env
URL: https://github.com/apache/doris-website/pull/60


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] zhannngchen commented on a diff in pull request #11953: [feature-wip](unique-key-merge-on-write) opt lock and only save valid rowsets delete_bitmap

[GitBox]

zhannngchen commented on code in PR #11953:
URL: https://github.com/apache/doris/pull/11953#discussion_r951105361


##
be/src/olap/tablet.h:
##
@@ -392,6 +393,9 @@ class Tablet : public BaseTablet {
 // TODO(lingbin): There is a _meta_lock TabletMeta too, there should be a 
comment to
 // explain how these two locks work together.
 mutable std::shared_mutex _meta_lock;
+
+// use to protect compaction and publish_txn update delete_bitmap
+mutable std::shared_mutex _delete_bitmap_lock;

Review Comment:
   rename to _rowset_update_mutex is better? And it should be a `mutex` rather 
than a `shared_mutex`.
   Also we need to add more comments on it:
   > In unique key table with MoW, we should guarantee that only one writer can 
update rowset and delete bitmap at the same time. We use a separate lock rather 
than `_meta_lock`, to avoid blocking read queries during publish_txn, which 
might take hundreds of milliseconds



##
be/src/olap/memtable.cpp:
##
@@ -400,20 +400,21 @@ bool MemTable::need_to_agg() {
 
 Status MemTable::_generate_delete_bitmap() {
 // generate delete bitmap, build a tmp rowset and load recent segment
-if (_tablet->enable_unique_key_merge_on_write()) {
-auto rowset = _rowset_writer->build_tmp();
-auto beta_rowset = reinterpret_cast(rowset.get());
-std::vector segments;
-segment_v2::SegmentSharedPtr segment;
-if (beta_rowset->num_segments() == 0) {
-return Status::OK();
-}
-RETURN_IF_ERROR(beta_rowset->load_segment(beta_rowset->num_segments() 
- 1, &segment));
-segments.push_back(segment);
-std::lock_guard 
meta_wrlock(_tablet->get_header_lock());
-RETURN_IF_ERROR(_tablet->calc_delete_bitmap(beta_rowset->rowset_id(), 
segments,
-&_rowset_ids, 
_delete_bitmap));
+if (!_tablet->enable_unique_key_merge_on_write()) {
+return Status::OK();
+}
+auto rowset = _rowset_writer->build_tmp();
+auto beta_rowset = reinterpret_cast(rowset.get());
+std::vector segments;
+segment_v2::SegmentSharedPtr segment;
+if (beta_rowset->num_segments() == 0) {
+return Status::OK();
 }
+RETURN_IF_ERROR(beta_rowset->load_segment(beta_rowset->num_segments() - 1, 
&segment));
+segments.push_back(segment);
+std::shared_lock meta_rlock(_tablet->get_header_lock());

Review Comment:
   Can we move this read lock into `calc_delete_bitmap`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] morrySnow commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

morrySnow commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951158836


##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/JoinLAsscomHelper.java:
##
@@ -165,7 +166,8 @@ private Pair, List> 
getProjectExprs() {
 private LogicalJoin newBottomJoin() {
 return new LogicalJoin(
 bottomJoin.getJoinType(),
-Optional.of(ExpressionUtils.and(newBottomJoinOnCondition)),
+newBottomJoinOnCondition,
+Optional.empty(),

Review Comment:
   why all other conjuncts are empty? do we will lose some expression on other 
conjuncts such as t1.a > t2.b?



##
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/exploration/join/JoinLAsscomProjectTest.java:
##
@@ -88,10 +88,12 @@ private Pair 
testJoinProjectLAsscom(List> project 
= new LogicalProject<>(
 projects,
-new LogicalJoin<>(JoinType.INNER_JOIN, 
Optional.of(bottomJoinOnCondition), scans.get(0), scans.get(1)));
+new LogicalJoin<>(JoinType.INNER_JOIN, 
Lists.newArrayList(bottomJoinOnCondition),
+Optional.empty(), scans.get(0), scans.get(1)));

Review Comment:
   we need to add case that join has other conjuncts



##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalJoin.java:
##
@@ -132,7 +165,10 @@ public List computeOutput(Plan leftInput, Plan 
rightInput) {
 @Override
 public String toString() {
 StringBuilder sb = new StringBuilder("LogicalJoin (").append(joinType);
-condition.ifPresent(expression -> sb.append(", ").append(expression));
+sb.append(" [");
+hashJoinConjuncts.stream().map(expr -> sb.append(" 
").append(expr)).collect(Collectors.toList());

Review Comment:
   please refer to PhysicalJoin's toString's comment



##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/logical/MultiJoin.java:
##
@@ -124,20 +140,27 @@ private Plan reorderJoinsAccordingToConditions(List 
joinInputs, List candidate.get(1));
-Set joinOutput = getJoinOutput(left, right);
-Map> split = splitConjuncts(conjuncts, 
joinOutput);
-List joinConditions = split.get(true);
-List nonJoinConditions = split.get(false);
-
-Optional cond;
+Pair, List> pair = 
JoinUtils.extractExpressionForHashTable(
+
joinInputs.get(0).getOutput().stream().map(SlotReference.class::cast).collect(Collectors.toList()),
+
joinInputs.get(1).getOutput().stream().map(SlotReference.class::cast).collect(Collectors.toList()),
+conjuncts);
+//Set joinOutput = getJoinOutput(left, right);
+//Map> split = splitConjuncts(conjuncts, 
joinOutput);
+//List joinConditions = split.get(true);
+//List nonJoinConditions = split.get(false);

Review Comment:
   remove it



##
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/exploration/join/JoinLAsscomTest.java:
##
@@ -73,9 +73,11 @@ public Pair testJoinLAsscom(
  */
 Assertions.assertEquals(3, scans.size());
 LogicalJoin bottomJoin = new 
LogicalJoin<>(JoinType.INNER_JOIN,
-Optional.of(bottomJoinOnCondition), scans.get(0), 
scans.get(1));
+Lists.newArrayList(bottomJoinOnCondition),

Review Comment:
   we need to add case that join has other conjuncts



##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalHashJoin.java:
##
@@ -65,7 +67,10 @@ public  R accept(PlanVisitor visitor, C context) 
{
 public String toString() {
 StringBuilder sb = new StringBuilder();
 sb.append("PhysicalHashJoin ([").append(joinType).append("]");
-condition.ifPresent(
+sb.append(" [");
+hashJoinConjuncts.stream().map(expr -> sb.append(" 
").append(expr)).collect(Collectors.toList());
+sb.append(" ]");

Review Comment:
   ```suggestion
   sb.append(Strings.join(hashJoinConjuncts, ", "));
   sb.append(" ]");
   ```



##
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/exploration/join/JoinCommuteTest.java:
##
@@ -47,7 +48,8 @@ public void testInnerJoinCommute() {
 new SlotReference("id", new BigIntType(), true, 
ImmutableList.of("table1")),
 new SlotReference("id", new BigIntType(), true, 
ImmutableList.of("table2")));
 LogicalJoin join = new LogicalJoin<>(
-JoinType.INNER_JOIN, Optional.of(onCondition), scan1, scan2);
+JoinType.INNER_JOIN, Lists.newArrayList(onCondition),

Review Comment:
   we need to add case that join has other conjuncts



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsub

[GitHub] [doris] fangzheng0518 commented on issue #9594: [Bug] PAUSE SYNC JOB 卡死

[GitBox]

fangzheng0518 commented on issue #9594:
URL: https://github.com/apache/doris/issues/9594#issuecomment-1222055644

   我也遇到了同样的问题，SYNC JOB里只有2张表（数据量才实时同步了2天，几万条数据而已）


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] Henry2SS opened a new issue, #11961: [Bug](error code) error code msg doesn't match usage

[GitBox]

Henry2SS opened a new issue, #11961:
URL: https://github.com/apache/doris/issues/11961

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Version
   
   trunk
   
   ### What's Wrong?
   
   Error msg is not right:
   
   `ERROR 1044 (42000): Access denied for user '%s'@'%s' to database '%s'`
   
   ### What You Expected?
   
   `ERROR 1044 (42000): Access denied for user 'user' to database 'test_db'`
   
   ### How to Reproduce?
   
   connecting a db using a user who doesn't have the access.
   
   ### Anything Else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] Henry2SS opened a new pull request, #11962: [Bug](error code) fix db access error code msg

[GitBox]

Henry2SS opened a new pull request, #11962:
URL: https://github.com/apache/doris/pull/11962

   # Proposed changes
   
   Issue Number: close #11961 
   
   ## Problem summary
   
   `ERR_DBACCESS_DENIED_ERROR` mismatch the usages.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [x] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [ ] No
   - [x] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [x] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [x] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [x] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] cambyzju closed pull request #11915: [fix](aggregate_node) _get_with_serialized_key_result deal with nullable

[GitBox]

cambyzju closed pull request #11915: [fix](aggregate_node) 
_get_with_serialized_key_result deal with nullable
URL: https://github.com/apache/doris/pull/11915


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] hf200012 opened a new pull request, #11963: [deps](httpv2)upgrade springboot version to 2.7.3

[GitBox]

hf200012 opened a new pull request, #11963:
URL: https://github.com/apache/doris/pull/11963

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [x] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [x] No
   - [ ] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [x] No
   - [ ] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [x] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [x] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] Kikyou1997 opened a new pull request, #11964: [feature](nerieds)Support partition prune

[GitBox]

Kikyou1997 opened a new pull request, #11964:
URL: https://github.com/apache/doris/pull/11964

   # Proposed changes
   
   Issue Number: noissue
   
   ## Problem summary
   
   Rewrite filter->project to project -> filter to make sure when do partition 
prune the tree looks like this: project -> filter -> olapscan.
   Enable the `MergeConsecutiveProject` `MergeConsecutiveFilter`.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [ ] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [ ] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [ ] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] yiguolei merged pull request #11955: [improvement](compress) Support compress/decompress block with lz4

[GitBox]

yiguolei merged PR #11955:
URL: https://github.com/apache/doris/pull/11955


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[doris] branch master updated: [improvement](compress) Support compress/decompress block with lz4 (#11955)

[yiguolei]

This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new c22d097b59 [improvement](compress) Support compress/decompress block 
with lz4 (#11955)
c22d097b59 is described below

commit c22d097b590870969175ed6face43c5e976ca2a7
Author: Jerry Hu 
AuthorDate: Mon Aug 22 17:35:43 2022 +0800

[improvement](compress) Support compress/decompress block with lz4 (#11955)
---
 be/src/runtime/runtime_state.h |  9 +
 .../aggregate_functions/aggregate_function_sort.h  |  3 +-
 be/src/vec/core/block.cpp  | 45 +++--
 be/src/vec/core/block.h| 10 +
 be/src/vec/runtime/vdata_stream_recvr.cpp  |  4 ++
 be/src/vec/runtime/vdata_stream_recvr.h|  2 +
 be/src/vec/sink/vdata_stream_sender.cpp|  8 +++-
 be/src/vec/sink/vdata_stream_sender.h  |  3 ++
 be/src/vec/sink/vtablet_sink.cpp   |  1 +
 be/test/vec/core/block_test.cpp| 46 +-
 .../java/org/apache/doris/qe/SessionVariable.java  | 10 +
 gensrc/proto/data.proto|  3 ++
 gensrc/thrift/PaloInternalService.thrift   |  2 +
 13 files changed, 113 insertions(+), 33 deletions(-)

diff --git a/be/src/runtime/runtime_state.h b/be/src/runtime/runtime_state.h
index f12f41ddee..033f19ae3f 100644
--- a/be/src/runtime/runtime_state.h
+++ b/be/src/runtime/runtime_state.h
@@ -351,6 +351,15 @@ public:
 return _query_options.enable_enable_exchange_node_parallel_merge;
 }
 
+segment_v2::CompressionTypePB fragement_transmission_compression_type() {
+if (_query_options.__isset.fragment_transmission_compression_codec) {
+if (_query_options.fragment_transmission_compression_codec == 
"lz4") {
+return segment_v2::CompressionTypePB::LZ4;
+}
+}
+return segment_v2::CompressionTypePB::SNAPPY;
+}
+
 // the following getters are only valid after Prepare()
 InitialReservations* initial_reservations() const { return 
_initial_reservations; }
 
diff --git a/be/src/vec/aggregate_functions/aggregate_function_sort.h 
b/be/src/vec/aggregate_functions/aggregate_function_sort.h
index ae320a6df9..2db72a4c5c 100644
--- a/be/src/vec/aggregate_functions/aggregate_function_sort.h
+++ b/be/src/vec/aggregate_functions/aggregate_function_sort.h
@@ -61,7 +61,8 @@ struct AggregateFunctionSortData {
 PBlock pblock;
 size_t uncompressed_bytes = 0;
 size_t compressed_bytes = 0;
-block.serialize(&pblock, &uncompressed_bytes, &compressed_bytes);
+block.serialize(&pblock, &uncompressed_bytes, &compressed_bytes,
+segment_v2::CompressionTypePB::SNAPPY);
 
 write_string_binary(pblock.SerializeAsString(), buf);
 }
diff --git a/be/src/vec/core/block.cpp b/be/src/vec/core/block.cpp
index 9aab2c311b..8728803a4d 100644
--- a/be/src/vec/core/block.cpp
+++ b/be/src/vec/core/block.cpp
@@ -34,6 +34,7 @@
 #include "runtime/tuple.h"
 #include "runtime/tuple_row.h"
 #include "udf/udf.h"
+#include "util/block_compression.h"
 #include "vec/columns/column.h"
 #include "vec/columns/column_const.h"
 #include "vec/columns/column_nullable.h"
@@ -79,16 +80,28 @@ Block::Block(const PBlock& pblock) {
 std::string compression_scratch;
 if (pblock.compressed()) {
 // Decompress
+SCOPED_RAW_TIMER(&_decompress_time_ns);
 const char* compressed_data = pblock.column_values().c_str();
 size_t compressed_size = pblock.column_values().size();
 size_t uncompressed_size = 0;
-bool success =
-snappy::GetUncompressedLength(compressed_data, 
compressed_size, &uncompressed_size);
-DCHECK(success) << "snappy::GetUncompressedLength failed";
-compression_scratch.resize(uncompressed_size);
-success =
-snappy::RawUncompress(compressed_data, compressed_size, 
compression_scratch.data());
-DCHECK(success) << "snappy::RawUncompress failed";
+if (pblock.has_compression_type() && pblock.has_uncompressed_size()) {
+std::unique_ptr codec;
+get_block_compression_codec(pblock.compression_type(), codec);
+uncompressed_size = pblock.uncompressed_size();
+compression_scratch.resize(uncompressed_size);
+Slice decompressed_slice(compression_scratch);
+codec->decompress(Slice(compressed_data, compressed_size), 
&decompressed_slice);
+DCHECK(uncompressed_size == decompressed_slice.size);
+} else {
+bool success = snappy::GetUncompressedLength(compressed_data, 
compressed_size,
+ &uncompressed_size);
+DCHECK(suc

[GitHub] [doris] caoliang-web opened a new pull request, #11965: [fix](MysqltoDoris)Failed to get doris_odbc_name value in mysql_to_doris, replace driver…

[GitBox]

caoliang-web opened a new pull request, #11965:
URL: https://github.com/apache/doris/pull/11965

   ## Problem summary
   
   1. Repair  Failed to get doris_odbc_name value in mysql_to_doris, replace 
driver value error
   2.mysql to doris document modification
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [ ] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [ ] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [ ] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951260479


##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalJoin.java:
##
@@ -132,7 +167,10 @@ public List computeOutput(Plan leftInput, Plan 
rightInput) {
 @Override
 public String toString() {
 StringBuilder sb = new StringBuilder("LogicalJoin (").append(joinType);
-condition.ifPresent(expression -> sb.append(", ").append(expression));
+sb.append(" [");
+hashJoinPredicates.stream().map(expr -> sb.append(" 
").append(expr)).collect(Collectors.toList());
+sb.append(" ]");
+otherJoinCondition.ifPresent(expression -> sb.append(", 
").append(expression));

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951262636


##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/AbstractPhysicalJoin.java:
##
@@ -78,11 +90,27 @@ public boolean equals(Object o) {
 return false;
 }
 AbstractPhysicalJoin that = (AbstractPhysicalJoin) o;
-return joinType == that.joinType && Objects.equals(condition, 
that.condition);
+return joinType == that.joinType && Objects.equals(otherJoinCondition, 
that.otherJoinCondition);

Review Comment:
   if there is hashJoinConjuncts like t1.x=t2.x, why choose NLJ, not hash join?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951266982


##
fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java:
##
@@ -51,14 +55,14 @@ public static boolean onlyShuffle(AbstractPhysicalJoin 
join) {
  */
 public static List getEqualTo(AbstractPhysicalJoin 
join) {
 List eqConjuncts = Lists.newArrayList();
-if (!join.getCondition().isPresent()) {
+if (!join.getOtherJoinCondition().isPresent()) {

Review Comment:
   This function is not used. It is totally replaced by 
extractExpressionForHashTable().  The purpose of this function is to get 
equations from otherJoinCondition.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951272217


##
fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java:
##
@@ -87,6 +91,117 @@ private static boolean isEqualTo(List 
leftSlots, List left;
+HashSet leftExprIds;
+HashSet right;
+HashSet rightExprIds;
+
+JoinSlotCoverageChecker(List left, List 
right) {
+this.left = new HashSet<>(left);
+leftExprIds = new 
HashSet<>(left.stream().map(SlotReference::getExprId).collect(Collectors.toList()));
+this.right = new HashSet<>(right);
+rightExprIds = new 
HashSet<>(right.stream().map(SlotReference::getExprId).collect(Collectors.toList()));
+}
+
+boolean isCoveredByLeftSlots(List slots) {
+boolean covered = left.containsAll(slots);

Review Comment:
   ok



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951275540


##
fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java:
##
@@ -98,19 +213,20 @@ public static Pair, 
List> getOnClauseUsedSlot
 
 List leftSlots = 
Utils.getOutputSlotReference(join.left());
 List rightSlots = 
Utils.getOutputSlotReference(join.right());
-List equalToList = getEqualTo(join);
-
+List equalToList = join.getHashJoinPredicates().stream()
+.map(e -> (EqualTo) e).collect(Collectors.toList());
+JoinSlotCoverageChecker checker = new 
JoinSlotCoverageChecker(leftSlots, rightSlots);
 for (EqualTo equalTo : equalToList) {
 List leftOnSlots = 
equalTo.left().collect(SlotReference.class::isInstance);
 List rightOnSlots = 
equalTo.right().collect(SlotReference.class::isInstance);
 
-if (new HashSet<>(leftSlots).containsAll(leftOnSlots)
-&& new HashSet<>(rightSlots).containsAll(rightOnSlots)) {
+if (checker.isCoveredByLeftSlots(leftOnSlots)
+&& checker.isCoveredByRightSlots(rightOnSlots)) {
 // TODO: need rethink about `.get(0)`
 childSlots.first.add(leftOnSlots.get(0));

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951276827


##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/algebra/Join.java:
##
@@ -28,5 +29,9 @@
 public interface Join {
 JoinType getJoinType();
 
-Optional getCondition();
+List getHashJoinPredicates();
+
+Optional getOtherJoinCondition();
+
+Optional getOnClauseCondition();

Review Comment:
   in current thrift, we have 3 types of condition, including where condition.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951277924


##
fe/fe-core/src/main/java/org/apache/doris/nereids/util/JoinUtils.java:
##
@@ -51,14 +55,14 @@ public static boolean onlyShuffle(AbstractPhysicalJoin 
join) {
  */
 public static List getEqualTo(AbstractPhysicalJoin 
join) {
 List eqConjuncts = Lists.newArrayList();
-if (!join.getCondition().isPresent()) {
+if (!join.getOtherJoinCondition().isPresent()) {
 return eqConjuncts;
 }
 
 List leftSlots = 
Utils.getOutputSlotReference(join.left());
 List rightSlots = 
Utils.getOutputSlotReference(join.right());
 
-Expression onCondition = join.getCondition().get();
+Expression onCondition = join.getOtherJoinCondition().get();

Review Comment:
   this function is not used. it is should be deleted.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] jacktengg opened a new issue, #11966: [Bug] ColumnDictionary::Dictionary::convert_code cause BE coredump

[GitBox]

jacktengg opened a new issue, #11966:
URL: https://github.com/apache/doris/issues/11966

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Version
   
   master 0c5b4ecc7
   
   ### What's Wrong?
   
   be coredump:
   ```
   *** Aborted at 1660976877 (unix time) try "date -d @1660976877" if you are 
using GNU date ***
   *** SIGSEGV address not mapped to object (@0x83) received by PID 217972 (TID 
0x7f4ff7f34700) from PID 131; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, 
siginfo_t*, void*) at 
/mnt/disk/tengjianping/doris-test/be/src/common/signal_handler.h:420
1# 0x7F500D06D400 in /lib64/libc.so.6
2# doris::vectorized::ColumnDictionary::Dictionary::convert_code(int 
const&) const at 
/mnt/disk/tengjianping/doris-test/be/src/vec/columns/column_dictionary.h:400
3# 
doris::vectorized::ColumnDictionary::convert_dict_codes_if_necessary() at 
/mnt/disk/tengjianping/doris-test/be/src/vec/columns/column_dictionary.h:242
4# doris::vectorized::ColumnNullable::convert_dict_codes_if_necessary() at 
/mnt/disk/tengjianping/doris-test/be/src/vec/columns/column_nullable.h:279
5# 
doris::segment_v2::SegmentIterator::_evaluate_short_circuit_predicate(unsigned 
short*, unsigned short*) at 
/mnt/disk/tengjianping/doris-test/be/src/olap/rowset/segment_v2/segment_iterator.cpp:877
6# 
doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) at 
/mnt/disk/tengjianping/doris-test/be/src/olap/rowset/segment_v2/segment_iterator.cpp:984
7# doris::BetaRowsetReader::next_block(doris::vectorized::Block*) at 
/mnt/disk/tengjianping/doris-test/be/src/olap/rowset/beta_rowset_reader.cpp:196
8# 
doris::vectorized::VCollectIterator::Level0Iterator::next(doris::vectorized::Block*)
 at /mnt/disk/tengjianping/doris-test/be/src/vec/olap/vcollect_iterator.cpp:230
9# 
doris::vectorized::VCollectIterator::Level1Iterator::_normal_next(doris::vectorized::Block*)
 at /mnt/disk/tengjianping/doris-test/be/src/vec/olap/vcollect_iterator.cpp:389
   10# 
doris::vectorized::VCollectIterator::Level1Iterator::_normal_next(doris::vectorized::Block*)
 at /mnt/disk/tengjianping/doris-test/be/src/vec/olap/vcollect_iterator.cpp:398
   11# 
doris::vectorized::VCollectIterator::Level1Iterator::next(doris::vectorized::Block*)
 at /mnt/disk/tengjianping/doris-test/be/src/vec/olap/vcollect_iterator.cpp:287
   12# doris::vectorized::VCollectIterator::next(doris::vectorized::Block*) at 
/mnt/disk/tengjianping/doris-test/be/src/vec/olap/vcollect_iterator.cpp:172
   13# 
doris::vectorized::BlockReader::_direct_next_block(doris::vectorized::Block*, 
doris::MemPool*, doris::ObjectPool*, bool*) at 
/mnt/disk/tengjianping/doris-test/be/src/vec/olap/block_reader.cpp:176
   14# 
doris::vectorized::BlockReader::next_block_with_aggregation(doris::vectorized::Block*,
 doris::MemPool*, doris::ObjectPool*, bool*) at 
/mnt/disk/tengjianping/doris-test/be/src/vec/olap/block_reader.h:50
   15# doris::vectorized::VOlapScanner::get_block(doris::RuntimeState*, 
doris::vectorized::Block*, bool*) at 
/mnt/disk/tengjianping/doris-test/be/src/vec/exec/volap_scanner.cpp:52
   16# 
doris::vectorized::VOlapScanNode::scanner_thread(doris::vectorized::VOlapScanner*)
 at /mnt/disk/tengjianping/doris-test/be/src/vec/exec/volap_scan_node.cpp:214
   17# void std::__invoke_impl(std::__invoke_memfun_deref, void 
(doris::vectorized::VOlapScanNode::*&)(doris::vectorized::VOlapScanner*), 
doris::vectorized::VOlapScanNode*&, doris::vectorized::VOlapScanner*&) at 
/mnt/disk/tengjianping/local/ldb_toolchain/include/c++/11/bits/invoke.h:74
   18# std::__invoke_result::type 
std::__invoke(void 
(doris::vectorized::VOlapScanNode::*&)(doris::vectorized::VOlapScanner*), 
doris::vectorized::VOlapScanNode*&, doris::vectorized::VOlapScanner*&) at 
/mnt/disk/tengjianping/local/ldb_toolchain/include/c++/11/bits/invoke.h:97
   19# void std::_Bind::__call(std::tuple<>&&, std::_Index_tuple<0ul, 1ul>) at 
/mnt/disk/tengjianping/local/ldb_toolchain/include/c++/11/functional:422
   20# void std::_Bind::operator()<,
 void>() at 
/mnt/disk/tengjianping/local/ldb_toolchain/include/c++/11/functional:505
   21# void std::__invoke_impl&>(std::__invoke_other,
 std::_Bind&) at 
/mnt/disk/tengjianping/local/ldb_toolchain/include/c++/11/bits/invoke.h:61
   22# std::enable_if&>, 
void>::type std::__invoke_r&>(std::_Bind&) at 
/mnt/disk/tengjianping/local/ldb_toolchain/include/c++/11/bits/invoke.h:117
   23# std::_Function_handler 
>::_M_invoke(std::_Any_data const&) at 
/mnt/disk/tengjianping/local/ldb_toolchain/include/c++/11/bits/std_function.h:292
   24# std::function::operator()() const at 
/mnt/disk/tengjianping/local/ldb_toolchain/include/c++/11/bits/std_function.h:560
   25# doris::PriorityWorkStealingThreadPool::work_thread(int) in 
/mnt/disk/tengjianping/doris-test/output/be/lib/doris_be
   26# vo

[GitHub] [doris] jacktengg opened a new pull request, #11967: [bugfix](dict) fix coredump of dict colum range predicate when there …

[GitBox]

jacktengg opened a new pull request, #11967:
URL: https://github.com/apache/doris/pull/11967

   …is null value
   
   # Proposed changes
   
   Issue Number: close #11966
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [ ] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [ ] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [ ] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] xinyiZzz opened a new issue, #11968: [Bug] DCHECK failed due to dependency on reservation tracker

[GitBox]

xinyiZzz opened a new issue, #11968:
URL: https://github.com/apache/doris/issues/11968

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Version
   
   192cdd4d760d9bef7669fd7811ff2153f979ee4f
   
   ### What's Wrong?
   
   Aborted at 1660936317 (unix time) try "date -d @1660936317" if you are using 
GNU date ***
   SIGABRT unkown detail explain (@0x15b35e) received by PID 1422174 (TID 
0x7f801e53f640) from PID 1422174; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, 
siginfo_t*, void*) at /doris/be/src/common/signal_handler.h:420
1# 0x7F8052F01040 in /lib/x86_64-linux-gnu/libc.so.6
2# raise in /lib/x86_64-linux-gnu/libc.so.6
3# abort in /lib/x86_64-linux-gnu/libc.so.6
4# google::ReadFromOffset(int, void*, unsigned long, long) [clone .cold] at 
src/symbolize.cc:146
5# 0x55C442312DBD at src/logging.cc:1650
6# google::LogMessage::SendToLog() at src/logging.cc:1607
7# google::LogMessage::Flush() at src/logging.cc:1477
8# google::LogMessageFatal::~LogMessageFatal() at src/logging.cc:2227
9# doris::PartitionedAggregationNode::open(doris::RuntimeState*) at 
/doris/be/src/exec/partitioned_aggregation_node.cc:286
   10# doris::PlanFragmentExecutor::open_internal() at 
/doris/be/src/runtime/plan_fragment_executor.cpp:370
   11# doris::PlanFragmentExecutor::open() at 
/doris/be/src/runtime/plan_fragment_executor.cpp:261
   12# doris::FragmentExecState::execute() at 
/doris/be/src/runtime/fragment_mgr.cpp:242
   13# 
doris::FragmentMgr::_exec_actual(std::shared_ptr, 
std::function) at 
/doris/be/src/runtime/fragment_mgr.cpp:482
   14# void std::_invoke_impl, 
std::function), doris::FragmentMgr*&, 
std::shared_ptr&, std::function&>(std::_invoke_memfun_deref, void 
(doris::FragmentMgr::&)(std::shared_ptr, 
std::function), doris::FragmentMgr*&, 
std::shared_ptr&, std::function&) in /be/lib/doris_be
   15# std::enable_if, 
std::function), doris::FragmentMgr*&, 
std::shared_ptr&, std::function&>, void>::type std::__invoke_r, 
std::function), doris::FragmentMgr*&, 
std::shared_ptr&, std::function&>(void 
(doris::FragmentMgr::&)(std::shared_ptr, 
std::function), doris::FragmentMgr*&, 
std::shared_ptr&, std::function&) in /be/lib/doris_be
   16# void std::Bind_result, std::function))(std::shared_ptr, 
std::function)>::_call(std::tuple<>&&, std::_Index_tuple<0ul, 1ul, 2ul>) in /be/lib/doris_be
   17# void std::_Bind_result, std::function))(std::shared_ptr, 
std::function)>::operator()<>() at 
/var/local/ldb_toolchain/include/c++/11/functional:631
   18# void std::_invoke_impl, std::function))(std::shared_ptr, 
std::function)>&>(std::_invoke_other, 
std::_Bind_result, std::function))(std::shared_ptr, 
std::function)>&) in /be/lib/doris_be
   19# std::enable_if, std::function))(std::shared_ptr, 
std::function)>&>, void>::type 
std::_invoke_rhttps://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] xinyiZzz opened a new pull request, #11969: [enhancement] Remove unused reservation tracker

[GitBox]

xinyiZzz opened a new pull request, #11969:
URL: https://github.com/apache/doris/pull/11969

   # Proposed changes
   
   Issue Number: close #11968
   
   ## Problem summary
   
   The reservation tracker is not actually used, resulting in:
   1. Uncontrollable DCHECK failure
   2. Exceed mem limit that is not easy to understand
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [ ] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [ ] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [ ] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951313629


##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/JoinLAsscomHelper.java:
##
@@ -165,7 +166,8 @@ private Pair, List> 
getProjectExprs() {
 private LogicalJoin newBottomJoin() {
 return new LogicalJoin(
 bottomJoin.getJoinType(),
-Optional.of(ExpressionUtils.and(newBottomJoinOnCondition)),
+newBottomJoinOnCondition,
+Optional.empty(),

Review Comment:
   In my understanding, LAsscom is used after FindHashConditionForJoin rule. 
LAsscom only consider `hashJoinCondition`. 
   for exmaple:
   ```
*  topJoinnewTopJoin
*  / \   / \
* bottomJoin  C   -->  newBottomJoin  B
*  /\  /\
* A  B  A  C
```
   The whole process is like this:
   1. collect otherJoinCondition of bottomJoin and topJoin in 
allOtherJoinCondition
   2. run LAsscom only consider hashJoinCondition
   3. push allOtherJoinCondition down to newTopJoin and newBottomJoin by 
predicatesPushdownThroughJoin rule.
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951313629


##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/JoinLAsscomHelper.java:
##
@@ -165,7 +166,8 @@ private Pair, List> 
getProjectExprs() {
 private LogicalJoin newBottomJoin() {
 return new LogicalJoin(
 bottomJoin.getJoinType(),
-Optional.of(ExpressionUtils.and(newBottomJoinOnCondition)),
+newBottomJoinOnCondition,
+Optional.empty(),

Review Comment:
   In my understanding, LAsscom is used after FindHashConditionForJoin rule. 
LAsscom only consider `hashJoinCondition`. 
   for exmaple:
   ```
*  topJoinnewTopJoin
*  /   \   / \
* bottomJoin  C   -->  newBottomJoin  B
*  /\  /\
* A  B  A  C
```
   The whole process is like this:
   1. collect otherJoinCondition of bottomJoin and topJoin in 
allOtherJoinCondition
   2. run LAsscom only consider hashJoinCondition
   3. push allOtherJoinCondition down to newTopJoin and newBottomJoin by 
predicatesPushdownThroughJoin rule.
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951313629


##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/JoinLAsscomHelper.java:
##
@@ -165,7 +166,8 @@ private Pair, List> 
getProjectExprs() {
 private LogicalJoin newBottomJoin() {
 return new LogicalJoin(
 bottomJoin.getJoinType(),
-Optional.of(ExpressionUtils.and(newBottomJoinOnCondition)),
+newBottomJoinOnCondition,
+Optional.empty(),

Review Comment:
   In my understanding, LAsscom is used after FindHashConditionForJoin rule. 
LAsscom only consider `hashJoinCondition`. 
   for exmaple:
   ```
  topJoinnewTopJoin
  / \ / \
 bottomJoin  C   -->  newBottomJoin  B
  /\  /\
 A  BA  C
```
   The whole process is like this:
   1. collect otherJoinCondition of bottomJoin and topJoin in 
allOtherJoinCondition
   2. run LAsscom only consider hashJoinCondition
   3. push allOtherJoinCondition down to newTopJoin and newBottomJoin by 
predicatesPushdownThroughJoin rule.
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] morrySnow commented on a diff in pull request #11867: [improve](nereids): enable exploration

[GitBox]

morrySnow commented on code in PR #11867:
URL: https://github.com/apache/doris/pull/11867#discussion_r951292608


##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/JoinCommute.java:
##
@@ -51,51 +54,44 @@ enum SwapType {
 
 @Override
 public Rule build() {
-return innerLogicalJoin(any(), any()).then(join -> {
-if (!check(join)) {
-return null;
-}
-boolean isBottomJoin = isBottomJoin(join);
-if (swapType == SwapType.BOTTOM_JOIN && !isBottomJoin) {
-return null;
-}
-
+return innerLogicalJoin().when(this::check).then(join -> {
 LogicalJoin newJoin = new LogicalJoin(
 join.getJoinType(),
 join.getCondition(),
 join.right(), join.left(),
-join.getJoinReorderContext()
-);
+join.getJoinReorderContext());
 newJoin.getJoinReorderContext().setHasCommute(true);
-if (swapType == SwapType.ZIG_ZAG && !isBottomJoin) {
-newJoin.getJoinReorderContext().setHasCommuteZigZag(true);
-}
+// if (swapType == SwapType.ZIG_ZAG && !isBottomJoin(join)) {

Review Comment:
   remove it or add a TODO to explain when to remove comment



##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/JoinCommute.java:
##
@@ -51,51 +54,44 @@ enum SwapType {
 
 @Override
 public Rule build() {
-return innerLogicalJoin(any(), any()).then(join -> {
-if (!check(join)) {
-return null;
-}
-boolean isBottomJoin = isBottomJoin(join);
-if (swapType == SwapType.BOTTOM_JOIN && !isBottomJoin) {
-return null;
-}
-
+return innerLogicalJoin().when(this::check).then(join -> {

Review Comment:
   remove any will lead to all children of join is GroupPlan and isBottomJoin 
will always return false



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] morningman merged pull request #11960: [Regression](rollup) Modify test case

[GitBox]

morningman merged PR #11960:
URL: https://github.com/apache/doris/pull/11960


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[doris] branch master updated (c22d097b59 -> 68e2b3db44)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


from c22d097b59 [improvement](compress) Support compress/decompress block 
with lz4 (#11955)
 add 68e2b3db44 [regression](rollup) Modify test case (#11960)

No new revisions were added by this update.

Summary of changes:
 regression-test/data/{rollup => rollup_p0}/test_rollup_agg_date.out   | 3 +--
 .../suites/{rollup => rollup_p0}/test_rollup_agg_date.groovy  | 4 ++--
 2 files changed, 3 insertions(+), 4 deletions(-)
 rename regression-test/data/{rollup => rollup_p0}/test_rollup_agg_date.out 
(91%)
 rename regression-test/suites/{rollup => 
rollup_p0}/test_rollup_agg_date.groovy (98%)


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951323386


##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/logical/MultiJoin.java:
##
@@ -124,20 +140,27 @@ private Plan reorderJoinsAccordingToConditions(List 
joinInputs, List candidate.get(1));
-Set joinOutput = getJoinOutput(left, right);
-Map> split = splitConjuncts(conjuncts, 
joinOutput);
-List joinConditions = split.get(true);
-List nonJoinConditions = split.get(false);
-
-Optional cond;
+Pair, List> pair = 
JoinUtils.extractExpressionForHashTable(
+
joinInputs.get(0).getOutput().stream().map(SlotReference.class::cast).collect(Collectors.toList()),
+
joinInputs.get(1).getOutput().stream().map(SlotReference.class::cast).collect(Collectors.toList()),
+conjuncts);
+//Set joinOutput = getJoinOutput(left, right);
+//Map> split = splitConjuncts(conjuncts, 
joinOutput);
+//List joinConditions = split.get(true);
+//List nonJoinConditions = split.get(false);

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] wsjz opened a new pull request, #11970: [feature-wip](parquet-reader) parquert scanner can read data

[GitBox]

wsjz opened a new pull request, #11970:
URL: https://github.com/apache/doris/pull/11970

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [ ] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [ ] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [ ] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] zenoyang commented on pull request #11967: [bugfix](dict) fix coredump of dict colum range predicate when there …

[GitBox]

zenoyang commented on PR #11967:
URL: https://github.com/apache/doris/pull/11967#issuecomment-171485

   How to reproduce the problem? I used the regression test case but it didn't 
reproduce.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] yixiutt commented on a diff in pull request #11953: [feature-wip](unique-key-merge-on-write) opt lock and only save valid rowsets delete_bitmap

[GitBox]

yixiutt commented on code in PR #11953:
URL: https://github.com/apache/doris/pull/11953#discussion_r951365833


##
be/src/olap/tablet.h:
##
@@ -392,6 +393,9 @@ class Tablet : public BaseTablet {
 // TODO(lingbin): There is a _meta_lock TabletMeta too, there should be a 
comment to
 // explain how these two locks work together.
 mutable std::shared_mutex _meta_lock;
+
+// use to protect compaction and publish_txn update delete_bitmap
+mutable std::shared_mutex _delete_bitmap_lock;

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] yixiutt commented on a diff in pull request #11953: [feature-wip](unique-key-merge-on-write) opt lock and only save valid rowsets delete_bitmap

[GitBox]

yixiutt commented on code in PR #11953:
URL: https://github.com/apache/doris/pull/11953#discussion_r951369397


##
be/src/olap/memtable.cpp:
##
@@ -400,20 +400,21 @@ bool MemTable::need_to_agg() {
 
 Status MemTable::_generate_delete_bitmap() {
 // generate delete bitmap, build a tmp rowset and load recent segment
-if (_tablet->enable_unique_key_merge_on_write()) {
-auto rowset = _rowset_writer->build_tmp();
-auto beta_rowset = reinterpret_cast(rowset.get());
-std::vector segments;
-segment_v2::SegmentSharedPtr segment;
-if (beta_rowset->num_segments() == 0) {
-return Status::OK();
-}
-RETURN_IF_ERROR(beta_rowset->load_segment(beta_rowset->num_segments() 
- 1, &segment));
-segments.push_back(segment);
-std::lock_guard 
meta_wrlock(_tablet->get_header_lock());
-RETURN_IF_ERROR(_tablet->calc_delete_bitmap(beta_rowset->rowset_id(), 
segments,
-&_rowset_ids, 
_delete_bitmap));
+if (!_tablet->enable_unique_key_merge_on_write()) {
+return Status::OK();
+}
+auto rowset = _rowset_writer->build_tmp();
+auto beta_rowset = reinterpret_cast(rowset.get());
+std::vector segments;
+segment_v2::SegmentSharedPtr segment;
+if (beta_rowset->num_segments() == 0) {
+return Status::OK();
 }
+RETURN_IF_ERROR(beta_rowset->load_segment(beta_rowset->num_segments() - 1, 
&segment));
+segments.push_back(segment);
+std::shared_lock meta_rlock(_tablet->get_header_lock());

Review Comment:
   publish_txn will also call this function , and code in publish_txn has some 
actions which should hold meta lock so I use lock outside



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] wangshuo128 commented on pull request #11964: [feature](nerieds)Support partition prune

[GitBox]

wangshuo128 commented on PR #11964:
URL: https://github.com/apache/doris/pull/11964#issuecomment-181377

   Please add unit tests to cover tests in `ListPartitionPrunerTest` and 
`RangePartitionPruneTest`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] starocean999 opened a new pull request, #11971: [fix](join)the hash join node would get the wrong nullable if the child node is cross join node

[GitBox]

starocean999 opened a new pull request, #11971:
URL: https://github.com/apache/doris/pull/11971

   
   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   cross join node has no output tuple for itself. Instead, use its children' 
tuple info instead. The hash join node set the output slot's nullable info 
based on its children' tuple info. So if the child is cross join node, hash 
join node should check if the cross join node's tuples is out joined, then can 
set the slots nullable info correctly.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [ ] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [ ] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [ ] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] jacktengg commented on pull request #11967: [bugfix](dict) fix coredump of dict colum range predicate when there …

[GitBox]

jacktengg commented on PR #11967:
URL: https://github.com/apache/doris/pull/11967#issuecomment-188069

   > 
   
   see issue


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] jacktengg commented on pull request #11967: [bugfix](dict) fix coredump of dict colum range predicate when there …

[GitBox]

jacktengg commented on PR #11967:
URL: https://github.com/apache/doris/pull/11967#issuecomment-188908

   > How to reproduce the problem? I used the regression test case but it 
didn't reproduce.
   
   See Reproduce in issue


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] zenoyang commented on pull request #11967: [bugfix](dict) fix coredump of dict colum range predicate when there …

[GitBox]

zenoyang commented on PR #11967:
URL: https://github.com/apache/doris/pull/11967#issuecomment-1222334729

   > > How to reproduce the problem? I used the regression test case but it 
didn't reproduce.
   > 
   > See Reproduce in issue
   
   https://user-images.githubusercontent.com/21984011/185928196-4c788e8c-6562-4324-b05a-70679b71f22b.png";>
   I used the master 0c5b4ec version, but still can't reproduce. 😅


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] BiteTheDDDDt opened a new pull request, #11972: [Enchancement](optimize) set result_size_hint to filter_block

[GitBox]

BiteThet opened a new pull request, #11972:
URL: https://github.com/apache/doris/pull/11972

   
   # Proposed changes
   
   ```sql
   SELECT SUM(LO_REVENUE - LO_SUPPLYCOST)
   FROM lineorder_flat
   WHERE
   C_REGION = 'AMERICA'
   AND S_REGION = 'AMERICA'
   AND( P_MFGR = 'MFGR#1'
   OR P_MFGR = 'MFGR#2');
   
   ```
   before:
   https://user-images.githubusercontent.com/7939630/185928793-ffaf10be-4fd2-4a0f-ae59-61e8624396f7.png";>
   after:
   https://user-images.githubusercontent.com/7939630/185928744-f793c22e-de2e-4e0b-8712-71a8647c2d85.png";>
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [ ] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [ ] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [ ] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] catpineapple opened a new pull request, #11973: [fix] prompt error when MySQL client login password is incorrect

[GitBox]

catpineapple opened a new pull request, #11973:
URL: https://github.com/apache/doris/pull/11973

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   prompt error when MySQL client login password is incorrect, because of 
multi-parameter error. 
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [ ✔️] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [ ] No
   - [ ✔️] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [✔️] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [✔️] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [✔️] No
   
   ## Further comments
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] hello-stephen opened a new pull request, #11974: [tools](ssb and tpch) optimize tools

[GitBox]

hello-stephen opened a new pull request, #11974:
URL: https://github.com/apache/doris/pull/11974

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   1. optimize the scripts in order to make them easier to use.
   2. use ShellCheck to format the code.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [*] No
   - [ ] I don't know
   4. Has unit tests been added:
   - [ ] Yes
   - [*] No
   - [ ] No Need
   5. Has document been added or modified:
   - [ ] Yes
   - [*] No
   - [ ] No Need
   6. Does it need to update dependencies:
   - [ ] Yes
   - [*] No
   7. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [*] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] hello-stephen opened a new pull request, #11975: [tools](ssb and tpch)optimize tools

[GitBox]

hello-stephen opened a new pull request, #11975:
URL: https://github.com/apache/doris/pull/11975

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   1. optimize the scripts in order to make them easier to use.
   2. use ShellCheck to format the code.
   3. use MYSQL_PWD instead of --password=xxx to suppress warning.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [ ] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   4. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   5. Does it need to update dependencies:
   - [ ] Yes
   - [ ] No
   6. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [ ] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] morrySnow closed pull request #11666: [feature](Nereids)support join algorithm in Nereids that already implemented in original planner

[GitBox]

morrySnow closed pull request #11666: [feature](Nereids)support join algorithm 
in Nereids that already implemented in original planner
URL: https://github.com/apache/doris/pull/11666


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] morrySnow opened a new pull request, #11976: [feature](Nereids)support join algorithm in Nereids that already implemented in original planner

[GitBox]

morrySnow opened a new pull request, #11976:
URL: https://github.com/apache/doris/pull/11976

   # Proposed changes
   
   - colocate join
   - bucket shuffle join
   - shuffle join
   - broadcast join
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [x] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [x] No
   - [ ] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [x] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [x] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [x] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #11967: [bugfix](dict) fix coredump of dict colum range predicate when there …

[GitBox]

github-actions[bot] commented on PR #11967:
URL: https://github.com/apache/doris/pull/11967#issuecomment-1222386041

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #11967: [bugfix](dict) fix coredump of dict colum range predicate when there …

[GitBox]

github-actions[bot] commented on PR #11967:
URL: https://github.com/apache/doris/pull/11967#issuecomment-1222386083

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] bin41215 opened a new pull request, #11977: [typo](doc) fix some typo in faq docs

[GitBox]

bin41215 opened a new pull request, #11977:
URL: https://github.com/apache/doris/pull/11977

   # Proposed changes
   fix some typo in faq docs
   
   ## Problem summary
   fix some typo in faq docs
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [x] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [ ] No
   - [x] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [x] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [x] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [x] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] pingchunzhang opened a new pull request, #11978: change community regression to different case level regression

[GitBox]

pingchunzhang opened a new pull request, #11978:
URL: https://github.com/apache/doris/pull/11978

   # Proposed changes
   
   Issue Number: NONE
   
   ## Problem summary
   
   change community regression to different case level regression for 2 purpose
   1、At present, the problem range of the pipeline is relatively wide. In order 
to further narrow the problem range, P0 runs basic cases with fast execution, 
and P1 runs simple cases with slightly slow execution
   2、To lay a foundation for more comprehensive testing in the future, P2 daily 
level execution cases will be added later
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] No
   2. Has unit tests been added:
   - [ ] No Need
   3. Has document been added or modified:
   - [ ] No Need
   4. Does it need to update dependencies:
   - [ ] No
   5. Are there any changes that cannot be rolled back:
   - [ ] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] jacktengg commented on pull request #11967: [bugfix](dict) fix coredump of dict colum range predicate when there …

[GitBox]

jacktengg commented on PR #11967:
URL: https://github.com/apache/doris/pull/11967#issuecomment-1222406111

   > > > How to reproduce the problem? I used the regression test case but it 
didn't reproduce.
   > > 
   > > 
   > > See Reproduce in issue
   > 
   > https://user-images.githubusercontent.com/21984011/185928196-4c788e8c-6562-4324-b05a-70679b71f22b.png";>
   > 
   > I used the master 
[0c5b4ec](https://github.com/apache/doris/commit/0c5b4ecc7cc46e2b014bb1f4c909be04f736c54a)
 version, but still can't reproduce. 😅
   
   Need enable vec engine adn storage vectorization. I just tried and can 
reproduce it...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] balan2307 commented on issue #11916: [Enhancement] replace `assertTrue(X.equals(X))` with `assertEquals`

[GitBox]

balan2307 commented on issue #11916:
URL: https://github.com/apache/doris/issues/11916#issuecomment-1222408434

   I am interested in solving this ,can you please assign me?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #11802: [feature](Nereids)add type coercion rule for nereids

[GitBox]

github-actions[bot] commented on PR #11802:
URL: https://github.com/apache/doris/pull/11802#issuecomment-1222436907

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #11802: [feature](Nereids)add type coercion rule for nereids

[GitBox]

github-actions[bot] commented on PR #11802:
URL: https://github.com/apache/doris/pull/11802#issuecomment-1222436977

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[doris] branch master updated: [FixAssist](compaction) add DCHECK in BlockReader::_unique_key_next_block to reason problem (#11951)

[dataroaring]

This is an automated email from the ASF dual-hosted git repository.

dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new b55195bd80 [FixAssist](compaction) add DCHECK in 
BlockReader::_unique_key_next_block to reason problem (#11951)
b55195bd80 is described below

commit b55195bd8026baab045165e99101c141f4875d61
Author: Yongqiang YANG <98214048+dataroar...@users.noreply.github.com>
AuthorDate: Mon Aug 22 22:33:31 2022 +0800

[FixAssist](compaction) add DCHECK in BlockReader::_unique_key_next_block 
to reason problem (#11951)
---
 be/src/vec/olap/block_reader.cpp | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/be/src/vec/olap/block_reader.cpp b/be/src/vec/olap/block_reader.cpp
index a42e7dbd56..cb1c1aa263 100644
--- a/be/src/vec/olap/block_reader.cpp
+++ b/be/src/vec/olap/block_reader.cpp
@@ -294,11 +294,13 @@ Status BlockReader::_unique_key_next_block(Block* block, 
MemPool* mem_pool, Obje
 filter_data[i] = delete_data[i] == 0;
 }
 
-ColumnWithTypeAndName column_with_type_and_name {
-_delete_filter_column, std::make_shared(), 
"filter"};
+ColumnWithTypeAndName column_with_type_and_name {_delete_filter_column,
+ 
std::make_shared(),
+ 
"__DORIS_COMPACTION_FILTER__"};
 block->insert(column_with_type_and_name);
 Block::filter_block(block, delete_sign_idx, target_columns.size());
 _stats.rows_del_filtered += target_block_row - block->rows();
+DCHECK(block->try_get_by_name("__DORIS_COMPACTION_FILTER__") == 
nullptr);
 }
 return Status::OK();
 }


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] dataroaring merged pull request #11951: [FixAssist](compaction) add DCHECK in BlockReader::_unique_key_next_block to reason problem

[GitBox]

dataroaring merged PR #11951:
URL: https://github.com/apache/doris/pull/11951


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951533772


##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/JoinLAsscomHelper.java:
##
@@ -165,7 +166,8 @@ private Pair, List> 
getProjectExprs() {
 private LogicalJoin newBottomJoin() {
 return new LogicalJoin(
 bottomJoin.getJoinType(),
-Optional.of(ExpressionUtils.and(newBottomJoinOnCondition)),
+newBottomJoinOnCondition,
+Optional.empty(),

Review Comment:
   add corresponding nonHashJoinConjuncts for bottom and top join.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #11978: change community regression to different case level regression

[GitBox]

github-actions[bot] commented on PR #11978:
URL: https://github.com/apache/doris/pull/11978#issuecomment-1222482732

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #11978: change community regression to different case level regression

[GitBox]

github-actions[bot] commented on PR #11978:
URL: https://github.com/apache/doris/pull/11978#issuecomment-1222482801

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] jackwener commented on a diff in pull request #11940: [refactor](status) Refactor status handling in agent task

[GitBox]

jackwener commented on code in PR #11940:
URL: https://github.com/apache/doris/pull/11940#discussion_r951545028


##
be/src/util/cpu_info.h:
##
@@ -20,12 +20,12 @@
 
 #pragma once
 
+#include 

Review Comment:
   Maybe it's redundant?



##
be/src/common/logconfig.cpp:
##
@@ -25,8 +25,7 @@
 #include 
 
 #include "common/config.h"
-#include "gutil/stringprintf.h"
-#include "util/logging.h"
+#include "common/logging.h"

Review Comment:
   Maybe it's redundant?



##
be/src/util/hash_util.hpp:
##
@@ -20,8 +20,9 @@
 
 #pragma once
 
+#include 

Review Comment:
   Maybe it's redundant?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] 924060929 merged pull request #11802: [feature](Nereids)add type coercion rule for nereids

[GitBox]

924060929 merged PR #11802:
URL: https://github.com/apache/doris/pull/11802


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] jackwener commented on pull request #11962: [Bug](error code) fix db access error code msg

[GitBox]

jackwener commented on PR #11962:
URL: https://github.com/apache/doris/pull/11962#issuecomment-1222492226

   please rebaes master to pass `P0`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] spaces-X commented on a diff in pull request #11972: [Enchancement](optimize) set result_size_hint to filter_block

[GitBox]

spaces-X commented on code in PR #11972:
URL: https://github.com/apache/doris/pull/11972#discussion_r951560496


##
be/src/vec/core/block.cpp:
##
@@ -638,7 +624,7 @@ void filter_block_internal(Block* block, const 
IColumn::Filter& filter, uint32_t
 if (count != block->rows()) {
 for (size_t i = 0; i < column_to_keep; ++i) {
 block->get_by_position(i).column =
-block->get_by_position(i).column->filter(filter, 0);
+block->get_by_position(i).column->filter(filter, 
count);

Review Comment:
   Please add UT for the new function call



##
be/src/vec/columns/columns_common.cpp:
##
@@ -106,14 +106,14 @@ struct ResultOffsetsBuilder {
 
 void insertOne(size_t array_size) {
 current_src_offset += array_size;
-res_offsets.push_back(current_src_offset);
+res_offsets.push_back_without_reserve(current_src_offset);
 }
 
 template 
 void insertChunk(const IColumn::Offset* src_offsets_pos, bool first,
  IColumn::Offset chunk_offset, size_t chunk_size) {
 const auto offsets_size_old = res_offsets.size();
-res_offsets.resize(offsets_size_old + SIMD_BYTES);
+res_offsets.resize_assume_reserved(offsets_size_old + SIMD_BYTES);

Review Comment:
   Make sure that `reserve()` function has been called before here in all cases.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #11977: [typo](doc) fix some typo in faq docs

[GitBox]

github-actions[bot] commented on PR #11977:
URL: https://github.com/apache/doris/pull/11977#issuecomment-1222496582

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #11977: [typo](doc) fix some typo in faq docs

[GitBox]

github-actions[bot] commented on PR #11977:
URL: https://github.com/apache/doris/pull/11977#issuecomment-1222496637

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #11940: [refactor](status) Refactor status handling in agent task

[GitBox]

github-actions[bot] commented on PR #11940:
URL: https://github.com/apache/doris/pull/11940#issuecomment-1222504470

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951570991


##
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/exploration/join/JoinCommuteTest.java:
##
@@ -47,7 +48,8 @@ public void testInnerJoinCommute() {
 new SlotReference("id", new BigIntType(), true, 
ImmutableList.of("table1")),
 new SlotReference("id", new BigIntType(), true, 
ImmutableList.of("table2")));
 LogicalJoin join = new LogicalJoin<>(
-JoinType.INNER_JOIN, Optional.of(onCondition), scan1, scan2);
+JoinType.INNER_JOIN, Lists.newArrayList(onCondition),

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951571270


##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalHashJoin.java:
##
@@ -65,7 +67,10 @@ public  R accept(PlanVisitor visitor, C context) 
{
 public String toString() {
 StringBuilder sb = new StringBuilder();
 sb.append("PhysicalHashJoin ([").append(joinType).append("]");
-condition.ifPresent(
+sb.append(" [");
+hashJoinConjuncts.stream().map(expr -> sb.append(" 
").append(expr)).collect(Collectors.toList());
+sb.append(" ]");

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] englefly commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

englefly commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951571537


##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalJoin.java:
##
@@ -132,7 +165,10 @@ public List computeOutput(Plan leftInput, Plan 
rightInput) {
 @Override
 public String toString() {
 StringBuilder sb = new StringBuilder("LogicalJoin (").append(joinType);
-condition.ifPresent(expression -> sb.append(", ").append(expression));
+sb.append(" [");
+hashJoinConjuncts.stream().map(expr -> sb.append(" 
").append(expr)).collect(Collectors.toList());

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #11962: [Bug](error code) fix db access error code msg

[GitBox]

github-actions[bot] commented on PR #11962:
URL: https://github.com/apache/doris/pull/11962#issuecomment-1222510578

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[doris] branch master updated: [feature](nereids): enable exploration job (#11867)

[huajianlan]

This is an automated email from the ASF dual-hosted git repository.

huajianlan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new def6f5568e [feature](nereids): enable exploration job (#11867)
def6f5568e is described below

commit def6f5568e690221e01a8c176c1d5cdd322f6b62
Author: jakevin 
AuthorDate: Mon Aug 22 23:38:17 2022 +0800

[feature](nereids): enable exploration job (#11867)

Enable the exploration job, and fix related problem.

correct the join reorder
---
 .../org/apache/doris/nereids/NereidsPlanner.java   |  2 +
 .../apache/doris/nereids/cost/CostEstimate.java|  2 +-
 .../glue/translator/PhysicalPlanTranslator.java|  7 
 .../jobs/cascades/ExploreGroupExpressionJob.java   |  5 +--
 .../jobs/cascades/OptimizeGroupExpressionJob.java  |  5 +--
 .../org/apache/doris/nereids/rules/RuleSet.java|  2 +-
 .../rules/exploration/join/JoinCommute.java| 46 ++
 .../nereids/trees/plans/logical/LogicalJoin.java   | 16 ++--
 8 files changed, 47 insertions(+), 38 deletions(-)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java
index a0a24f9f66..5be733f91c 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/NereidsPlanner.java
@@ -121,6 +121,8 @@ public class NereidsPlanner extends Planner {
 
 // Get plan directly. Just for SSB.
 PhysicalPlan physicalPlan = getRoot().extractPlan();
+// TODO: remove above
+// PhysicalPlan physicalPlan = chooseBestPlan(getRoot(), 
PhysicalProperties.ANY);
 
 // post-process physical plan out of memo, just for future use.
 return postprocess(physicalPlan);
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/cost/CostEstimate.java 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/cost/CostEstimate.java
index da60840877..a25d725706 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/nereids/cost/CostEstimate.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/cost/CostEstimate.java
@@ -35,7 +35,7 @@ public final class CostEstimate {
  * Constructor of CostEstimate.
  */
 public CostEstimate(double cpuCost, double memoryCost, double networkCost) 
{
-// TODO: remove them after finish statistics.
+// TODO: fix stats
 if (cpuCost < 0) {
 cpuCost = 0;
 }
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java
 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java
index f560adf8eb..56490dd0a0 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java
@@ -45,6 +45,7 @@ import org.apache.doris.nereids.trees.plans.PlanType;
 import org.apache.doris.nereids.trees.plans.logical.LogicalSort;
 import org.apache.doris.nereids.trees.plans.physical.AbstractPhysicalSort;
 import org.apache.doris.nereids.trees.plans.physical.PhysicalAggregate;
+import org.apache.doris.nereids.trees.plans.physical.PhysicalDistribution;
 import org.apache.doris.nereids.trees.plans.physical.PhysicalFilter;
 import org.apache.doris.nereids.trees.plans.physical.PhysicalHashJoin;
 import org.apache.doris.nereids.trees.plans.physical.PhysicalLimit;
@@ -493,6 +494,12 @@ public class PhysicalPlanTranslator extends 
DefaultPlanVisitor 
distribution,
+PlanTranslatorContext context) {
+return distribution.child().accept(this, context);
+}
+
 private void extractExecSlot(Expr root, Set slotRefList) {
 if (root instanceof SlotRef) {
 slotRefList.add(((SlotRef) root).getDesc().getId().asInt());
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/cascades/ExploreGroupExpressionJob.java
 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/cascades/ExploreGroupExpressionJob.java
index a74b588ff7..6a7a5e59d0 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/cascades/ExploreGroupExpressionJob.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/cascades/ExploreGroupExpressionJob.java
@@ -25,8 +25,6 @@ import org.apache.doris.nereids.memo.GroupExpression;
 import org.apache.doris.nereids.pattern.Pattern;
 import org.apache.doris.nereids.rules.Rule;
 
-import com.google.common.collect.Lists;
-
 import java.util.Comparator;
 import java.util.List;
 
@@ -50,8 +48,7 @@ public class ExploreGroupExpressionJob extends Job {
 @Override
 public void execute() {
 // TODO: enable exploration job after we test it
-// List> explorationRules = 
getRuleSet().getExplorationRules();
-  

[GitHub] [doris] 924060929 merged pull request #11867: [feature](nereids): enable exploration job

[GitBox]

924060929 merged PR #11867:
URL: https://github.com/apache/doris/pull/11867


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] morningman opened a new pull request, #11979: [feature](http) get create table stmt for a given query

[GitBox]

morningman opened a new pull request, #11979:
URL: https://github.com/apache/doris/pull/11979

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   This API can help user to get all create table statement of a given SQL.
   So that it will be easier to run the case in other Doris cluster.
   See document for details
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [ ] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [ ] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [ ] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] morningman merged pull request #11978: [github](checks) change the requirement of github checks

[GitBox]

morningman merged PR #11978:
URL: https://github.com/apache/doris/pull/11978


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[doris] branch master updated: [github](checks) change the requirement of github checks (#11978)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 38c751e5eb [github](checks) change the requirement of github checks 
(#11978)
38c751e5eb is described below

commit 38c751e5ebe1811cc3df7d72c7d88408293592a2
Author: chunping 
AuthorDate: Tue Aug 23 00:01:05 2022 +0800

[github](checks) change the requirement of github checks (#11978)
---
 .asf.yaml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.asf.yaml b/.asf.yaml
index d3f49b8a62..b192a0295d 100644
--- a/.asf.yaml
+++ b/.asf.yaml
@@ -51,7 +51,8 @@ github:
   - License Check
   - Clang Formatter
   - CheckStyle
-  - P0 regression (Doris P0 Regression)
+  - P0 Regression (Doris Regression)
+  - P1 Regression (Doris Regression)
   - FE UT (Doris FE UT)
   - BE UT (Doris BE UT)
   - Build Broker


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #11973: [fix] prompt error when MySQL client login password is incorrect

[GitBox]

github-actions[bot] commented on PR #11973:
URL: https://github.com/apache/doris/pull/11973#issuecomment-1222621235

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #11973: [fix] prompt error when MySQL client login password is incorrect

[GitBox]

github-actions[bot] commented on PR #11973:
URL: https://github.com/apache/doris/pull/11973#issuecomment-1222621289

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] morrySnow commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

morrySnow commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r95172


##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/logical/MultiJoin.java:
##
@@ -51,11 +54,27 @@ public class MultiJoin extends PlanVisitor {
  *A  B
  */
 public final List joinInputs = new ArrayList<>();
-public final List conjuncts = new ArrayList<>();
+public final List conjunctsForAllHashJoins = new ArrayList<>();
 
+private List conjunctsKeepInFilter = new ArrayList<>();

Review Comment:
   got it



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] morrySnow commented on a diff in pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

morrySnow commented on code in PR #11812:
URL: https://github.com/apache/doris/pull/11812#discussion_r951670737


##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/JoinLAsscomHelper.java:
##
@@ -74,10 +78,20 @@ public JoinLAsscomHelper(LogicalJoin topJoin,
 b = bottomJoin.right();
 c = topJoin.right();
 
-Preconditions.checkArgument(topJoin.getCondition().isPresent(), 
"topJoin onClause must be present.");
-topJoinOnClause = topJoin.getCondition().get();
-Preconditions.checkArgument(bottomJoin.getCondition().isPresent(), 
"bottomJoin onClause must be present.");
-bottomJoinOnClause = bottomJoin.getCondition().get();
+Preconditions.checkArgument(!topJoin.getHashJoinConjuncts().isEmpty(),
+"topJoin hashJoinConjuncts must exist.");
+topHashJoinConjuncts = topJoin.getHashJoinConjuncts();
+if (topJoin.getOtherJoinCondition().isPresent()) {
+allNonHashJoinConjuncts.addAll(
+
ExpressionUtils.extractConjunction(topJoin.getOtherJoinCondition().get()));
+}
+
Preconditions.checkArgument(!bottomJoin.getHashJoinConjuncts().isEmpty(),
+"bottomJoin onClause must exist.");

Review Comment:
   nit: put all check at the begin to avoid use them before check 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] github-actions[bot] commented on pull request #11812: [feature](Nereids) support non-equal predicates in Join

[GitBox]

github-actions[bot] commented on PR #11812:
URL: https://github.com/apache/doris/pull/11812#issuecomment-1222642906

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] xinyiZzz opened a new pull request, #11980: [dev-1.1.2](memtracker) Fix DCHECK consumption is greater than 0 in old mem tracker

[GitBox]

xinyiZzz opened a new pull request, #11980:
URL: https://github.com/apache/doris/pull/11980

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   ```
   #0  0x7f00aa2f2fbb in raise () from /lib/x86_64-linux-gnu/libc.so.6
   #1  0x7f00aa2d8864 in abort () from /lib/x86_64-linux-gnu/libc.so.6
   #2  0x55797ba76bf2 in google::DumpStackTraceAndExit () at 
src/utilities.cc:160
   #3  0x5579824a62bd in google::LogMessage::Fail () at src/logging.cc:1650
   #4  0x5579824a87f9 in google::LogMessage::SendToLog 
(this=0x7f0066e6f2a0) at src/logging.cc:1607
   #5  0x5579824a5e26 in google::LogMessage::Flush (this=0x7f0066e6f2a0) at 
src/logging.cc:1476
   #6  0x5579824a8e69 in google::LogMessageFatal::~LogMessageFatal 
(this=, __in_chrg=) at src/logging.cc:2226
   #7  0x55797c4f6a8b in doris::MemTracker::Release (this=0x613000200380, 
bytes=8192) at /doris/be/src/runtime/mem_tracker.h:253
   #8  0x55797d090315 in doris::RowBatch::clear (this=0x614000314c40) at 
/doris/be/src/runtime/row_batch.cpp:230
   #9  0x55797d0904c1 in doris::RowBatch::~RowBatch (this=0x614000314c40, 
__in_chrg=) at /doris/be/src/runtime/row_batch.cpp:236
   #10 0x55797d090576 in doris::RowBatch::~RowBatch (this=0x614000314c40, 
__in_chrg=) at /doris/be/src/runtime/row_batch.cpp:237
   #11 0x55797d08261f in std::default_delete::operator() 
(this=0x62100287ee98, __ptr=0x614000314c40) at 
/include/c++/11/bits/unique_ptr.h:85
   #12 0x55797d082de8 in std::__uniq_ptr_impl >::reset (this=0x62100287ee98, __p=0x0)
   at /include/c++/11/bits/unique_ptr.h:182
   #13 0x55797d081022 in std::unique_ptr >::reset (this=0x62100287ee98,
   __p=) at /include/c++/11/bits/unique_ptr.h:456
   #14 0x55797d07b065 in doris::PlanFragmentExecutor::close 
(this=0x62100287e570) at /doris/be/src/runtime/plan_fragment_executor.cpp:668
   #15 0x55797ce43727 in doris::FragmentExecState::execute 
(this=0x62100287e500) at /doris/be/src/runtime/fragment_mgr.cpp:244
   #16 0x55797ce4adb3 in 
doris::FragmentMgr::_exec_actual(std::shared_ptr, 
std::function) (this=0x61413c40,
   exec_state=std::shared_ptr (use count 3, weak 
count 0) = {...}, cb=...) at /doris/be/src/runtime/fragment_mgr.cpp:482
   ```
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [ ] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [ ] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [ ] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] xinyiZzz commented on issue #10400: [Bug]

[GitBox]

xinyiZzz commented on issue #10400:
URL: https://github.com/apache/doris/issues/10400#issuecomment-1222783309

   错误原因是超出了query内存限制，doris中一个query默认执行内存限制为2G，
   修改session variable `set exec_mem_limit = ;` 提高当前session的query内存限制，
   也可以 `set global exec_mem_limit = ;` 修改query默认内存限制，重启mysql client生效。


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] xinyiZzz closed issue #10400: [Bug]

[GitBox]

xinyiZzz closed issue #10400: [Bug] 
URL: https://github.com/apache/doris/issues/10400


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] xinyiZzz closed issue #6680: [Bug] 当并行执行sql语句BE占用内存达到mem_limit上限值时，BE进程异常退出

[GitBox]

xinyiZzz closed issue #6680: [Bug] 当并行执行sql语句BE占用内存达到mem_limit上限值时，BE进程异常退出
URL: https://github.com/apache/doris/issues/6680


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] xinyiZzz commented on issue #6680: [Bug] 当并行执行sql语句BE占用内存达到mem_limit上限值时，BE进程异常退出

[GitBox]

xinyiZzz commented on issue #6680:
URL: https://github.com/apache/doris/issues/6680#issuecomment-1222790934

   @dh-cloud @freshsun @chovy-3012 
   The old version of the query mem tracker is inaccurate and cannot 
effectively limit the memory usage of the BE process.
   
   After 1.1.2, and the latest master, a new mem tracker is implemented, which 
can effectively limit BE memory usage and avoid OOM
   
   1.1.2 will be released soon, welcome to try the updated version


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] AshinGau opened a new pull request, #11981: [feature-wip](parquet-reader) parquet dictionary decoder

[GitBox]

AshinGau opened a new pull request, #11981:
URL: https://github.com/apache/doris/pull/11981

   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem summary
   
   Describe your changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: 
   - [ ] Yes
   - [ ] No
   - [ ] I don't know
   2. Has unit tests been added:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   3. Has document been added or modified:
   - [ ] Yes
   - [ ] No
   - [ ] No Need
   4. Does it need to update dependencies:
   - [ ] Yes
   - [ ] No
   5. Are there any changes that cannot be rolled back:
   - [ ] Yes (If Yes, please explain WHY)
   - [ ] No
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] s11ngh commented on issue #11916: [Enhancement] replace `assertTrue(X.equals(X))` with `assertEquals`

[GitBox]

s11ngh commented on issue #11916:
URL: https://github.com/apache/doris/issues/11916#issuecomment-1223056259

   Hi, could you assign this issue? Would love working on it


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[doris] branch dev-1.1.2 updated: [Bugfix](inpredicate) fix in predicate in group by clause may cause NPE (#11886)

[yiguolei]

This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch dev-1.1.2
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/dev-1.1.2 by this push:
 new 0eb51086b9 [Bugfix](inpredicate) fix in predicate in group by clause 
may cause NPE (#11886)
0eb51086b9 is described below

commit 0eb51086b98c608f5448258411aa867c5b2fa911
Author: Zhengguo Yang 
AuthorDate: Sun Aug 21 10:03:30 2022 +0800

[Bugfix](inpredicate) fix in predicate in group by clause may cause NPE 
(#11886)

* [bug](inpredicate) fix in predicate in group by clause may cause NPE
---
 .../java/org/apache/doris/rewrite/RewriteInPredicateRule.java | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/rewrite/RewriteInPredicateRule.java 
b/fe/fe-core/src/main/java/org/apache/doris/rewrite/RewriteInPredicateRule.java
index b37377e72a..b4ec881a1b 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/rewrite/RewriteInPredicateRule.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/rewrite/RewriteInPredicateRule.java
@@ -54,8 +54,12 @@ public class RewriteInPredicateRule implements 
ExprRewriteRule {
 }
 InPredicate inPredicate = (InPredicate) expr;
 SlotRef slotRef;
-if (inPredicate.contains(Subquery.class) || 
!inPredicate.isLiteralChildren() || inPredicate.isNotIn()
-|| !(inPredicate.getChild(0).unwrapExpr(false) instanceof 
SlotRef)
+// When the select stmt contains group by, we use oriGroupingExprs to 
store the original group by statement
+// and reset it with the rewritten groupingExpr. Therefore, 
origroupingexprs cannot be analyzed.
+// However, in #4197, oriGroupingExprs is rewritten to fix the problem 
of constant fold.
+// The newly added InPredicteRewriteRule requires that expr must be 
analyzed before being rewritten
+if (!inPredicate.isAnalyzed() || inPredicate.contains(Subquery.class) 
|| !inPredicate.isLiteralChildren()
+|| inPredicate.isNotIn() || 
!(inPredicate.getChild(0).unwrapExpr(false) instanceof SlotRef)
 || (slotRef = inPredicate.getChild(0).getSrcSlotRef()) == null 
|| slotRef.getColumn() == null) {
 return expr;
 }


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[doris] branch dev-1.1.2 updated: [fix](agg)repeat node shouldn't change slot's nullable property of agg node (#11859)

[yiguolei]

This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch dev-1.1.2
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/dev-1.1.2 by this push:
 new e158613189 [fix](agg)repeat node shouldn't change slot's nullable 
property of agg node (#11859)
e158613189 is described below

commit e158613189058d98fb08bb2917daf48399393fc0
Author: starocean999 <40539150+starocean...@users.noreply.github.com>
AuthorDate: Mon Aug 22 16:28:45 2022 +0800

[fix](agg)repeat node shouldn't change slot's nullable property of agg node 
(#11859)
---
 fe/fe-core/src/main/java/org/apache/doris/planner/RepeatNode.java | 8 
 1 file changed, 8 deletions(-)

diff --git a/fe/fe-core/src/main/java/org/apache/doris/planner/RepeatNode.java 
b/fe/fe-core/src/main/java/org/apache/doris/planner/RepeatNode.java
index 61c212b9e4..b9f6eeec16 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/planner/RepeatNode.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/planner/RepeatNode.java
@@ -24,10 +24,8 @@ import org.apache.doris.analysis.GroupByClause;
 import org.apache.doris.analysis.GroupingInfo;
 import org.apache.doris.analysis.SlotDescriptor;
 import org.apache.doris.analysis.SlotId;
-import org.apache.doris.analysis.SlotRef;
 import org.apache.doris.analysis.TupleDescriptor;
 import org.apache.doris.analysis.TupleId;
-import org.apache.doris.analysis.VirtualSlotRef;
 import org.apache.doris.common.UserException;
 import org.apache.doris.thrift.TExplainLevel;
 import org.apache.doris.thrift.TPlanNode;
@@ -122,12 +120,6 @@ public class RepeatNode extends PlanNode {
 
outputTupleDesc.setTable(analyzer.getTupleDesc(inputTupleIds.get(0)).getTable());
 }
 
-//set aggregate nullable
-for (Expr slot : groupByClause.getGroupingExprs()) {
-if (slot instanceof SlotRef && !(slot instanceof VirtualSlotRef)) {
-((SlotRef) slot).getDesc().setIsNullable(true);
-}
-}
 outputTupleDesc.computeStatAndMemLayout();
 
 List> groupingIdList = new ArrayList<>();


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] yiguolei merged pull request #11582: [feature-wip](new-scan) add scanner scheduling framework

[GitBox]

yiguolei merged PR #11582:
URL: https://github.com/apache/doris/pull/11582


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[doris] branch master updated: [feature-wip](new-scan) add scanner scheduling framework (#11582)

[yiguolei]

This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
 new 05da3d947f [feature-wip](new-scan) add scanner scheduling framework 
(#11582)
05da3d947f is described below

commit 05da3d947f63766b2ac2d6e430a4b2467ea17042
Author: Mingyu Chen 
AuthorDate: Tue Aug 23 08:45:18 2022 +0800

[feature-wip](new-scan) add scanner scheduling framework (#11582)

There are currently many types of ScanNodes in Doris. And most of the logic 
of these ScanNodes is the same, including:

Runtime filter
Predicate pushdown
Scanner generation and scheduling
So I intend to unify the common logic of all ScanNodes.
Different data sources only need to implement different Scanners for data 
access.
So that the future optimization for scan can be applied to the scan of all 
data sources,
while also reducing the code duplication.

This PR mainly adds 4 new class:

VScanner
All Scanners' parent class. The subclasses can inherit this class to 
implement specific data access methods.

VScanNode
The unified ScanNode, and is responsible for common logic including 
RuntimeFilter, predicate pushdown, Scanner generation and scheduling.

ScannerContext
ScannerContext is responsible for recording the execution status
of a group of Scanners corresponding to a ScanNode.
Including how many scanners are being scheduled, and maintaining
a producer-consumer blocks queue between scanners and scan nodes.

ScannerContext is also the scheduling unit of ScannerScheduler.
ScannerScheduler schedules a ScannerContext at a time,
and submits the Scanners to the scanner thread pool for data scanning.

ScannerScheduler
Unified responsible for all Scanner scheduling tasks

Test:
This work is still in progress and default is disabled.
I tested it with jmeter with 50 concurrency, but currently the scanner is 
just return without data.
The QPS can reach about 9000.
I can't compare it to origin implement because no data is read for now. I 
will test it when new olap scanner is ready.
Co-authored-by: morningman 
---
 be/src/common/config.h |   4 +
 be/src/exec/exec_node.cpp  |  34 +-
 be/src/exec/olap_scan_node.cpp |   8 +-
 be/src/runtime/exec_env.h  |   5 +-
 be/src/runtime/exec_env_init.cpp   |   6 +
 be/src/runtime/fragment_mgr.cpp|  25 +-
 be/src/runtime/plan_fragment_executor.cpp  |  20 +-
 be/src/runtime/query_fragments_ctx.h   |  22 +-
 be/src/vec/CMakeLists.txt  |  13 +-
 be/src/vec/exec/scan/new_olap_scan_node.cpp| 307 
 be/src/vec/exec/scan/new_olap_scan_node.h  |  64 ++
 be/src/vec/exec/scan/new_olap_scanner.cpp  | 325 
 be/src/vec/exec/scan/new_olap_scanner.h|  79 ++
 be/src/vec/exec/scan/scanner_context.cpp   | 232 ++
 be/src/vec/exec/scan/scanner_context.h | 205 +
 be/src/vec/exec/scan/scanner_scheduler.cpp | 253 ++
 be/src/vec/exec/scan/scanner_scheduler.h   |  94 +++
 be/src/vec/exec/scan/vscan_node.cpp| 864 +
 be/src/vec/exec/scan/vscan_node.h  | 246 ++
 be/src/vec/exec/scan/vscanner.cpp  | 155 
 be/src/vec/exec/scan/vscanner.h| 140 
 be/src/vec/exec/volap_scan_node.cpp|   7 +-
 be/src/vec/olap/block_reader.cpp   |   3 +
 be/src/vec/sink/vdata_stream_sender.h  |   2 +-
 .../org/apache/doris/regression/Config.groovy  |   2 +-
 25 files changed, 3060 insertions(+), 55 deletions(-)

diff --git a/be/src/common/config.h b/be/src/common/config.h
index 44749c095c..10aeeff6d5 100644
--- a/be/src/common/config.h
+++ b/be/src/common/config.h
@@ -840,6 +840,10 @@ CONF_Int32(doris_remote_scanner_thread_pool_thread_num, 
"16");
 // number of s3 scanner thread pool queue size
 CONF_Int32(doris_remote_scanner_thread_pool_queue_size, "10240");
 
+// If set to true, the new scan node framework will be used.
+// This config should be removed when the new scan node is ready.
+CONF_Bool(enable_new_scan_node, "false");
+
 #ifdef BE_TEST
 // test s3
 CONF_String(test_s3_resource, "resource");
diff --git a/be/src/exec/exec_node.cpp b/be/src/exec/exec_node.cpp
index 5c4981c739..66827019f9 100644
--- a/be/src/exec/exec_node.cpp
+++ b/be/src/exec/exec_node.cpp
@@ -62,6 +62,7 @@
 #include "vec/core/block.h"
 #include "vec/exec/file_scan_node.h"
 #include "vec/exec/join/vhash_join_node.h"
+#include "vec/exec/scan/new_olap_scan_node.h"
 #include "vec/exec/vaggregation_node.h"
 #include "vec/exec/vanalytic_eval_node.h"
 #inclu

[GitHub] [doris] morningman commented on a diff in pull request #11952: [improvement](storage) For debugging problems: add session variable to treat agg and unique data model as dup model

[GitBox]

morningman commented on code in PR #11952:
URL: https://github.com/apache/doris/pull/11952#discussion_r952040885


##
fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java:
##
@@ -525,6 +529,18 @@ public class SessionVariable implements Serializable, 
Writable {
 @VariableMgr.VarAttr(name = ENABLE_FUNCTION_PUSHDOWN)
 public boolean enableFunctionPushdown;
 
+/**
+ * For debugg purpose, dont' merge unique key and agg key when reading 
data.
+ */
+@VariableMgr.VarAttr(name = SKIP_STORAGE_ENGINE_MERGE)
+private boolean skipStorageEngineMerge = false;
+
+/**
+ * For debugg purpose, skip delte predicate when reading data.
+ */
+@VariableMgr.VarAttr(name = SKIP_DELETE_PREDICATE)
+private boolean skipDeletePredicate = false;

Review Comment:
   use `public`, or it may cause some serde probem.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[doris] branch dev-1.1.2 updated: [fix][Vectorized] Fix nullptr deref in data sink (#11473)

[yiguolei]

This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch dev-1.1.2
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/dev-1.1.2 by this push:
 new 4a2b4d05b3 [fix][Vectorized] Fix nullptr deref in data sink (#11473)
4a2b4d05b3 is described below

commit 4a2b4d05b31b56d4388dd170314708111a82d3bd
Author: Amos Bird 
AuthorDate: Mon Aug 22 11:44:55 2022 +0800

[fix][Vectorized] Fix nullptr deref in data sink (#11473)

brpc cache may return nullptr.
---
 be/src/vec/sink/vdata_stream_sender.cpp | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/be/src/vec/sink/vdata_stream_sender.cpp 
b/be/src/vec/sink/vdata_stream_sender.cpp
index 0f1cf57c7f..416e96954e 100644
--- a/be/src/vec/sink/vdata_stream_sender.cpp
+++ b/be/src/vec/sink/vdata_stream_sender.cpp
@@ -68,6 +68,13 @@ Status VDataStreamSender::Channel::init(RuntimeState* state) 
{
 _brpc_stub = 
state->exec_env()->brpc_internal_client_cache()->get_client(_brpc_dest_addr);
 }
 
+if (!_brpc_stub) {
+std::string msg = fmt::format("Get rpc stub failed, dest_addr={}:{}",
+  _brpc_dest_addr.hostname, 
_brpc_dest_addr.port);
+LOG(WARNING) << msg;
+return Status::InternalError(msg);
+}
+
 // In bucket shuffle join will set fragment_instance_id (-1, -1)
 // to build a camouflaged empty channel. the ip and port is '0.0.0.0:0"
 // so the empty channel not need call function close_internal()


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] morningman commented on a diff in pull request #11952: [improvement](storage) For debugging problems: add session variable to treat agg and unique data model as dup model

[GitBox]

morningman commented on code in PR #11952:
URL: https://github.com/apache/doris/pull/11952#discussion_r952042390


##
fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java:
##
@@ -525,6 +529,18 @@ public class SessionVariable implements Serializable, 
Writable {
 @VariableMgr.VarAttr(name = ENABLE_FUNCTION_PUSHDOWN)
 public boolean enableFunctionPushdown;
 
+/**
+ * For debugg purpose, dont' merge unique key and agg key when reading 
data.
+ */
+@VariableMgr.VarAttr(name = SKIP_STORAGE_ENGINE_MERGE)
+private boolean skipStorageEngineMerge = false;

Review Comment:
   If this is true, can we select __DELETE_SIGN__ directly?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[GitHub] [doris] yiguolei merged pull request #11974: [tools](ssb and tpch) optimize tools

[GitBox]

yiguolei merged PR #11974:
URL: https://github.com/apache/doris/pull/11974


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org

[doris] branch dev-1.1.2 updated: [tools](ssb and tpch) optimize tools (#11974)

[yiguolei]

This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch dev-1.1.2
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/dev-1.1.2 by this push:
 new 68cdc20fc8 [tools](ssb and tpch) optimize tools (#11974)
68cdc20fc8 is described below

commit 68cdc20fc8499b485b1f6c1676fcb29dfc16afbd
Author: Dongyang Li 
AuthorDate: Tue Aug 23 09:10:49 2022 +0800

[tools](ssb and tpch) optimize tools (#11974)
---
 tools/ssb-tools/README.md  |  40 +++
 tools/ssb-tools/{ => bin}/build-ssb-dbgen.sh   |  21 +-
 .../bin/create-ssb-tables.sh}  |  24 +-
 tools/ssb-tools/{ => bin}/gen-ssb-data.sh  |  72 --
 tools/ssb-tools/bin/load-ssb-data.sh   | 276 +
 .../bin/run-ssb-flat-queries.sh}   |  49 ++--
 .../bin/run-ssb-queries.sh}|  44 ++--
 tools/ssb-tools/{ => conf}/doris-cluster.conf  |   6 +-
 tools/ssb-tools/ddl/create-ssb-flat-table.sql  |  74 ++
 .../create-ssb-tables.sql} | 115 +
 tools/ssb-tools/load-dimension-data.sh |  68 -
 tools/ssb-tools/load-fact-data.sh  | 126 --
 .../q1.3.sql => ssb-flat-queries/q1.1.sql} |  17 +-
 .../{queries => ssb-flat-queries}/q1.2.sql |  16 +-
 .../{queries => ssb-flat-queries}/q1.3.sql |  18 +-
 .../q1.2.sql => ssb-flat-queries/q2.1.sql} |  16 +-
 .../q1.1.sql => ssb-flat-queries/q2.2.sql} |  19 +-
 .../q1.1.sql => ssb-flat-queries/q2.3.sql} |  18 +-
 .../q1.1.sql => ssb-flat-queries/q3.1.sql} |  21 +-
 .../q1.1.sql => ssb-flat-queries/q3.2.sql} |  21 +-
 .../q1.1.sql => ssb-flat-queries/q3.3.sql} |  21 +-
 .../q1.1.sql => ssb-flat-queries/q3.4.sql} |  21 +-
 .../q1.1.sql => ssb-flat-queries/q4.1.sql} |  19 +-
 .../q1.1.sql => ssb-flat-queries/q4.2.sql} |  25 +-
 .../q2.1.sql => ssb-flat-queries/q4.3.sql} |  23 +-
 tools/ssb-tools/{queries => ssb-queries}/q1.1.sql  |  15 +-
 tools/ssb-tools/{queries => ssb-queries}/q1.2.sql  |  15 +-
 tools/ssb-tools/{queries => ssb-queries}/q1.3.sql  |  18 +-
 tools/ssb-tools/{queries => ssb-queries}/q2.1.sql  |  16 +-
 tools/ssb-tools/{queries => ssb-queries}/q2.2.sql  |  17 +-
 tools/ssb-tools/{queries => ssb-queries}/q2.3.sql  |  16 +-
 tools/ssb-tools/{queries => ssb-queries}/q3.1.sql  |  26 +-
 tools/ssb-tools/{queries => ssb-queries}/q3.2.sql  |  26 +-
 tools/ssb-tools/{queries => ssb-queries}/q3.3.sql  |  34 ++-
 tools/ssb-tools/{queries => ssb-queries}/q3.4.sql  |  33 ++-
 tools/ssb-tools/{queries => ssb-queries}/q4.1.sql  |  30 ++-
 tools/ssb-tools/{queries => ssb-queries}/q4.2.sql  |  36 +--
 tools/ssb-tools/{queries => ssb-queries}/q4.3.sql  |  30 ++-
 tools/tpch-tools/README.md |  25 +-
 tools/tpch-tools/{ => bin}/build-tpch-dbgen.sh |  17 +-
 tools/tpch-tools/{ => bin}/create-tpch-tables.sh   |  12 +-
 tools/tpch-tools/{ => bin}/gen-tpch-data.sh|  32 +--
 tools/tpch-tools/{ => bin}/load-tpch-data.sh   |  97 
 tools/tpch-tools/{ => bin}/run-tpch-queries.sh |  36 +--
 tools/tpch-tools/{ => conf}/doris-cluster.conf |   4 +-
 tools/tpch-tools/create-tpch-tables.sql| 100 
 tools/tpch-tools/ddl/create-tpch-tables.sql| 174 +
 tools/tpch-tools/queries/q1.sql|   2 +-
 tools/tpch-tools/queries/q10.sql   |  22 +-
 tools/tpch-tools/queries/q11.sql   |  27 +-
 tools/tpch-tools/queries/q12.sql   |   2 +-
 tools/tpch-tools/queries/q13.sql   |   6 +-
 tools/tpch-tools/queries/q14.sql   |   8 +-
 tools/tpch-tools/queries/q15.sql   |  17 +-
 tools/tpch-tools/queries/q16.sql   |   2 +-
 tools/tpch-tools/queries/q17.sql   |  20 +-
 tools/tpch-tools/queries/q18.sql   |  58 +++--
 tools/tpch-tools/queries/q19.sql   |   2 +-
 tools/tpch-tools/queries/q2.sql|  44 ++--
 tools/tpch-tools/queries/q20.sql   |  62 ++---
 tools/tpch-tools/queries/q21.sql   |  66 +++--
 tools/tpch-tools/queries/q22.sql   |   2 +-
 tools/tpch-tools/queries/q3.sql|  22 +-
 tools/tpch-tools/queries/q4.sql|  21 +-
 tools/tpch-tools/queries/q5.sql|   2 +-
 tools/tpch-tools/queries/q6.sql|   2 +-
 tools/tpch-tools/queries/q7.sql|   2 +-
 tools/tpch-tools/queries/q8.sql|   8 +-
 tools/tpch-tools/queries/q9.sql|  26 +-
 69 files changed, 1426 insertions(+), 976 deletions(-)

diff --git a/tools/ssb-tools/README.md b/tools/ssb-tools/README.md
new file mode 100644
index 00..6d8e85e97d
--- /dev/nul

[GitHub] [doris] yiguolei merged pull request #11973: [fix](error-code) prompt error when MySQL client login password is incorrect

[GitBox]

yiguolei merged PR #11973:
URL: https://github.com/apache/doris/pull/11973


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org