[GitHub] [doris] yiguolei merged pull request #12095: [fix](planner) fix orthogonal_bitmap_union_count plan : wrong PREAGGREGATION
yiguolei merged PR #12095: URL: https://github.com/apache/doris/pull/12095 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (461a4cc94e -> 491dd34ba7)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 461a4cc94e [Enhancement](Error Msg) show details of COLUMN and TABLE name regex #11999 add 491dd34ba7 [fix](planner) fix orthogonal_bitmap_union_count plan : wrong PREAGGREGATION (#12095) No new revisions were added by this update. Summary of changes: .../apache/doris/planner/SingleNodePlanner.java| 30 -- .../org/apache/doris/planner/QueryPlanTest.java| 23 + 2 files changed, 40 insertions(+), 13 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #12417: [docs](quick-compaction) enable quick compaction by default
yiguolei merged PR #12417: URL: https://github.com/apache/doris/pull/12417 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (491dd34ba7 -> 0ea7c4b37b)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 491dd34ba7 [fix](planner) fix orthogonal_bitmap_union_count plan : wrong PREAGGREGATION (#12095) add 0ea7c4b37b [docs](quick-compaction) update quick-compaction docs (#12417) No new revisions were added by this update. Summary of changes: docs/en/docs/admin-manual/config/be-config.md| 2 +- docs/zh-CN/docs/admin-manual/config/be-config.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on a diff in pull request #12444: [feature](nereids)add syntax support for double/float
Kikyou1997 commented on code in PR #12444:
URL: https://github.com/apache/doris/pull/12444#discussion_r965577862
##
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##
@@ -962,4 +965,9 @@ public Expression visitExist(ExistContext context) {
public List withInList(PredicateContext ctx) {
return
ctx.expression().stream().map(this::getExpression).collect(ImmutableList.toImmutableList());
}
+
+@Override
+public DecimalLiteral visitDecimalLiteral(DecimalLiteralContext ctx) {
+return new DecimalLiteral(new BigDecimal(ctx.getText()));
Review Comment:
Nah, just keep same as stale parser.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #12437: [fix](agg) crash caused by failure of prepare
yiguolei merged PR #12437: URL: https://github.com/apache/doris/pull/12437 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](agg) crash caused by failure of prepare (#12437)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 14221adbbd [fix](agg) crash caused by failure of prepare (#12437)
14221adbbd is described below
commit 14221adbbd931a84cf6c9707589927d557fc835b
Author: Jerry Hu
AuthorDate: Thu Sep 8 15:03:45 2022 +0800
[fix](agg) crash caused by failure of prepare (#12437)
---
be/src/vec/exec/vaggregation_node.cpp | 24 ++--
1 file changed, 18 insertions(+), 6 deletions(-)
diff --git a/be/src/vec/exec/vaggregation_node.cpp
b/be/src/vec/exec/vaggregation_node.cpp
index b8a3705c96..71f06deb24 100644
--- a/be/src/vec/exec/vaggregation_node.cpp
+++ b/be/src/vec/exec/vaggregation_node.cpp
@@ -90,7 +90,16 @@ AggregationNode::AggregationNode(ObjectPool* pool, const
TPlanNode& tnode,
_build_timer(nullptr),
_serialize_key_timer(nullptr),
_exec_timer(nullptr),
- _merge_timer(nullptr) {
+ _merge_timer(nullptr),
+ _expr_timer(nullptr),
+ _get_results_timer(nullptr),
+ _serialize_data_timer(nullptr),
+ _serialize_result_timer(nullptr),
+ _deserialize_data_timer(nullptr),
+ _hash_table_compute_timer(nullptr),
+ _streaming_agg_timer(nullptr),
+ _hash_table_size_counter(nullptr),
+ _hash_table_input_counter(nullptr) {
if (tnode.agg_node.__isset.use_streaming_preaggregation) {
_is_streaming_preagg = tnode.agg_node.use_streaming_preaggregation;
if (_is_streaming_preagg) {
@@ -499,11 +508,14 @@ Status AggregationNode::close(RuntimeState* state) {
VExpr::close(_probe_expr_ctxs, state);
if (_executor.close) _executor.close();
-std::visit(
-[&](auto&& agg_method) {
-COUNTER_SET(_hash_table_size_counter,
int64_t(agg_method.data.size()));
-},
-_agg_data._aggregated_method_variant);
+/// _hash_table_size_counter may be null if prepare failed.
+if (_hash_table_size_counter) {
+std::visit(
+[&](auto&& agg_method) {
+COUNTER_SET(_hash_table_size_counter,
int64_t(agg_method.data.size()));
+},
+_agg_data._aggregated_method_variant);
+}
return ExecNode::close(state);
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangshuo128 opened a new pull request, #12464: [Feature](Nereids) Main framework for selecting rollup index.
wangshuo128 opened a new pull request, #12464: URL: https://github.com/apache/doris/pull/12464 # Proposed changes First step of #12303 ## Problem summary This is the first step for supporting rollup index selection for aggregate/unique key OLAP table. This PR aims to select rollup index when the aggregate node is present and the aggregate function matches the value type. So pre-aggregation is turned on by default. Cases that pre-aggregation should be turned off will be addressed in the next PR. Main steps for rollup index selection: 1. filter rollup indexes with all the required columns. 2. filter rollup indexes that match the key prefix most. 3. order the rollup indexes by row count, column count, rollup index id. TODO remaining: 1. address cases that pre-aggregation should be turned off. (next PR) 2. add more test cases. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 5. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 6. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 7. Does it need to update dependencies: - [ ] Yes - [ ] No 8. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] TaoZex opened a new issue, #12465: [Feature] The field function support like mysql.
TaoZex opened a new issue, #12465: URL: https://github.com/apache/doris/issues/12465 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description Need poseexplode function support, this function has been implemented in MySQL. ### Use case https://www.w3schools.com/sql/func_mysql_field.asp You can view the usage of the field function on this page. ### Related issues _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener merged pull request #12439: [feature](Nereids): Left deep tree join order.
jackwener merged PR #12439: URL: https://github.com/apache/doris/pull/12439 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [feature](Nereids): Left deep tree join order. (#12439)
This is an automated email from the ASF dual-hosted git repository.
jakevin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 7c7ac86fe8 [feature](Nereids): Left deep tree join order. (#12439)
7c7ac86fe8 is described below
commit 7c7ac86fe8e3360e39b96d2c517aa25b61b18426
Author: jakevin
AuthorDate: Thu Sep 8 15:09:22 2022 +0800
[feature](Nereids): Left deep tree join order. (#12439)
* [feature](Nereids): Left deep tree join order.
---
.../org/apache/doris/nereids/rules/RuleSet.java| 44 +++-
.../rules/exploration/join/JoinCommute.java| 74 +++---
.../rules/exploration/join/JoinCommuteProject.java | 66 --
.../rules/exploration/join/JoinLAsscom.java| 50 +++--
.../rules/exploration/join/JoinLAsscomHelper.java | 250 +
.../rules/exploration/join/JoinLAsscomProject.java | 55 +++--
...inCommuteHelper.java => JoinReorderCommon.java} | 36 ++-
.../exploration/{ => join}/JoinReorderContext.java | 19 +-
.../rules/exploration/join/ThreeJoinHelper.java| 165 ++
.../nereids/trees/plans/logical/LogicalJoin.java | 2 +-
.../apache/doris/nereids/util/ExpressionUtils.java | 22 ++
.../rules/exploration/join/JoinCommuteTest.java| 30 ++-
.../exploration/join/JoinLAsscomProjectTest.java | 134 ---
.../rules/exploration/join/JoinLAsscomTest.java| 156 +
.../org/apache/doris/nereids/util/PlanChecker.java | 37 ++-
15 files changed, 548 insertions(+), 592 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/RuleSet.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/RuleSet.java
index fb2696788c..936834605b 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/RuleSet.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/rules/RuleSet.java
@@ -18,7 +18,8 @@
package org.apache.doris.nereids.rules;
import org.apache.doris.nereids.rules.exploration.join.JoinCommute;
-import org.apache.doris.nereids.rules.exploration.join.JoinCommuteProject;
+import org.apache.doris.nereids.rules.exploration.join.JoinLAsscom;
+import org.apache.doris.nereids.rules.exploration.join.JoinLAsscomProject;
import
org.apache.doris.nereids.rules.implementation.LogicalAggToPhysicalHashAgg;
import
org.apache.doris.nereids.rules.implementation.LogicalAssertNumRowsToPhysicalAssertNumRows;
import
org.apache.doris.nereids.rules.implementation.LogicalEmptyRelationToPhysicalEmptyRelation;
@@ -32,6 +33,7 @@ import
org.apache.doris.nereids.rules.implementation.LogicalProjectToPhysicalPro
import
org.apache.doris.nereids.rules.implementation.LogicalSortToPhysicalQuickSort;
import org.apache.doris.nereids.rules.implementation.LogicalTopNToPhysicalTopN;
import org.apache.doris.nereids.rules.rewrite.AggregateDisassemble;
+import org.apache.doris.nereids.rules.rewrite.logical.MergeConsecutiveProjects;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableList.Builder;
@@ -43,8 +45,10 @@ import java.util.List;
*/
public class RuleSet {
public static final List EXPLORATION_RULES = planRuleFactories()
-.add(JoinCommute.SWAP_OUTER_SWAP_ZIG_ZAG)
-.add(JoinCommuteProject.SWAP_OUTER_SWAP_ZIG_ZAG)
+.add(JoinCommute.OUTER_LEFT_DEEP)
+.add(JoinLAsscom.INNER)
+.add(JoinLAsscomProject.INNER)
+.add(new MergeConsecutiveProjects())
.build();
public static final List REWRITE_RULES = planRuleFactories()
@@ -66,6 +70,40 @@ public class RuleSet {
.add(new LogicalEmptyRelationToPhysicalEmptyRelation())
.build();
+public static final List LEFT_DEEP_TREE_JOIN_REORDER =
planRuleFactories()
+.add(JoinCommute.OUTER_LEFT_DEEP)
+.add(JoinLAsscom.INNER)
+.add(JoinLAsscomProject.INNER)
+.add(JoinLAsscom.OUTER)
+.add(JoinLAsscomProject.OUTER)
+// semi join Transpose
+.build();
+
+public static final List ZIG_ZAG_TREE_JOIN_REORDER =
planRuleFactories()
+.add(JoinCommute.OUTER_ZIG_ZAG)
+.add(JoinLAsscom.INNER)
+.add(JoinLAsscomProject.INNER)
+.add(JoinLAsscom.OUTER)
+.add(JoinLAsscomProject.OUTER)
+// semi join Transpose
+.build();
+
+public static final List BUSHY_TREE_JOIN_REORDER =
planRuleFactories()
+.add(JoinCommute.OUTER_BUSHY)
+// TODO: add more rule
+// .add(JoinLeftAssociate.INNER)
+// .add(JoinLeftAssociateProject.INNER)
+// .add(JoinRightAssociate.INNER)
+// .add(JoinRightAssociateProject.INNER)
+// .add(JoinExchange.INNER)
+// .add(JoinExchangeBothProject.INNER)
+// .add(JoinExchangeLeftProject.INNER)
+
[GitHub] [doris] github-actions[bot] commented on pull request #12460: [fix](stream load) Fix wrong conversion of null value when vstream load json format
github-actions[bot] commented on PR #12460: URL: https://github.com/apache/doris/pull/12460#issuecomment-1240318775 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #12460: [fix](stream load) Fix wrong conversion of null value when vstream load json format
github-actions[bot] commented on PR #12460: URL: https://github.com/apache/doris/pull/12460#issuecomment-1240318819 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] 924060929 closed pull request #12462: [feature](Nereids) Support invoke exploration rule in PlanChecker and add an example
924060929 closed pull request #12462: [feature](Nereids) Support invoke exploration rule in PlanChecker and add an example URL: https://github.com/apache/doris/pull/12462 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #12441: [fix](agg)the intermediate slots should be materialized as output slots
morrySnow commented on code in PR #12441:
URL: https://github.com/apache/doris/pull/12441#discussion_r965591870
##
fe/fe-core/src/main/java/org/apache/doris/analysis/AggregateInfo.java:
##
@@ -834,6 +834,31 @@ public void updateTypeOfAggregateExprs() {
}
}
+public void updateMaterializedSlots() {
+// why output and intermediate may have different materialized slots?
+// because some slot is materialized by materializeSrcExpr method
directly
+// in that case, only output slots is materialized
+// assume output tuple has correct marterialized infomation
+// we update intermediate tuple and materializedSlots based on output
tuple
+materializedSlots.clear();
+ArrayList outputSlots = outputTupleDesc.getSlots();
+int groupingExprNum = groupingExprs != null ? groupingExprs.size() : 0;
+Preconditions.checkState(groupingExprNum <= outputSlots.size());
+for (int i = groupingExprNum; i < outputSlots.size(); ++i) {
+if (outputSlots.get(i).isMaterialized()) {
+materializedSlots.add(i - groupingExprNum);
+}
+}
+outputTupleDesc.computeStatAndMemLayout();
Review Comment:
why need yo re-compute output's layout?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangbo opened a new issue, #12466: [Bug] 1.1 version Column prune incorrect result
wangbo opened a new issue, #12466: URL: https://github.com/apache/doris/issues/12466 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version 1.1 ### What's Wrong? wrong result ### What You Expected? fix it ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] caoliang-web opened a new issue, #12467: [Bug] select into outfile export format is parquet failed
caoliang-web opened a new issue, #12467: URL: https://github.com/apache/doris/issues/12467 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version 1.1.2 ### What's Wrong? Use select into outfile to export the format to parquet. When vectorization is turned on, the error parquet writer not support will appear. vectorization turned off export is ok ### What You Expected? Use select into outfile to export the format to parquet. When vectorization is turned on, it can also be exported normally ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 opened a new pull request, #12468: [feature-wip](nereids) implement uncheckedCast method in VarcharLiteral
Kikyou1997 opened a new pull request, #12468: URL: https://github.com/apache/doris/pull/12468 # Proposed changes Issue Number: noissue ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on pull request #12468: [feature-wip](nereids) implement uncheckedCast method in VarcharLiteral
Kikyou1997 commented on PR #12468: URL: https://github.com/apache/doris/pull/12468#issuecomment-1240331292 @morrySnow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener opened a new pull request, #12469: [improve](Nereids): check same logicalProperty when insert a Group.
jackwener opened a new pull request, #12469: URL: https://github.com/apache/doris/pull/12469 # Proposed changes Issue Number: close #xxx ## Problem summary check same logicalProperty when insert a Group. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] deardeng opened a new pull request, #12470: [fix] Fix notify update storage policy agent task null exception
deardeng opened a new pull request, #12470: URL: https://github.com/apache/doris/pull/12470 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [x] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [x] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No Fix cold_heat_separation regression case java.lang.NullPointerException such as ``` 2022-09-08 01:09:39,705 WARN (agent-task-pool-5|289) [AgentBatchTask.run():176] task exec error. backend[10003] java.lang.NullPointerException: null at org.apache.doris.thrift.BackendService$submit_tasks_args$submit_tasks_argsStandardScheme.write(BackendService.java:6456) ~[spark-dpp-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] at org.apache.doris.thrift.BackendService$submit_tasks_args$submit_tasks_argsStandardScheme.write(BackendService.java:6404) ~[spark-dpp-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] at org.apache.doris.thrift.BackendService$submit_tasks_args.write(BackendService.java:6358) ~[spark-dpp-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:71) ~[spark-dpp-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62) ~[spark-dpp-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] at org.apache.doris.thrift.BackendService$Client.sendSubmitTasks(BackendService.java:227) ~[spark-dpp-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] at org.apache.doris.thrift.BackendService$Client.submitTasks(BackendService.java:219) ~[spark-dpp-1.0-SNAPSHOT.jar:1.0-SNAPSHOT] at org.apache.doris.task.AgentBatchTask.run(AgentBatchTask.java:167) ~[doris-fe.jar:1.0-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] qidaye commented on a diff in pull request #12401: [Enhancement](DOE): Doe support object/nested use string
qidaye commented on code in PR #12401:
URL: https://github.com/apache/doris/pull/12401#discussion_r965604625
##
fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsUtil.java:
##
@@ -358,22 +358,23 @@ public static List genColumnsFromEs(EsRestClient
client, String indexNam
List columns = new ArrayList<>();
for (String key : keys) {
JSONObject field = (JSONObject) mappingProps.get(key);
+Type type;
// Complex types are not currently supported.
Review Comment:
```suggestion
// Complex types are treating as STRING types for now.
```
##
fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/MappingPhase.java:
##
@@ -119,6 +119,8 @@ private static void resolveDocValuesFields(SearchContext
searchContext, JSONObje
if (!docValue) {
return;
}
+} else if (fieldType == null) {
Review Comment:
Add some comments to explain why we need this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #12159: [feature](nereids)support distinct count
morrySnow commented on code in PR #12159:
URL: https://github.com/apache/doris/pull/12159#discussion_r965606605
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/AggregateFunction.java:
##
@@ -25,13 +25,24 @@
public abstract class AggregateFunction extends BoundFunction {
private DataType intermediate;
+private final boolean isDistinct;
Review Comment:
need overload equals and hashcode and add ut to ExpressionEqualsTest
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalAggregate.java:
##
@@ -57,24 +57,32 @@
private final List outputExpressions;
private final AggPhase aggPhase;
+// use for scenes containing distinct agg
+// 1. If there are LOCAL and GLOBAL phases, global is the final phase
+// 2. If there are LOCAL, GLOBAL and DISTINCT_LOCAL phases, DISTINCT_LOCAL
is the final phase
+// 3. If there are LOCAL, GLOBAL, DISTINCT_LOCAL, DISTINCT_GLOBAL phases,
+// DISTINCT_GLOBAL is the final phase
+private final boolean isFinalPhase;
+
/**
* Desc: Constructor for LogicalAggregate.
*/
public LogicalAggregate(
List groupByExpressions,
List outputExpressions,
CHILD_TYPE child) {
-this(groupByExpressions, outputExpressions, false, false,
AggPhase.GLOBAL, child);
+this(groupByExpressions, outputExpressions, false, false, false,
AggPhase.GLOBAL, child);
Review Comment:
i think the default of finalize should be true and aggphase should set to
local to support one stage aggregate
##
fe/fe-core/src/main/java/org/apache/doris/nereids/properties/RequestPropertyDeriver.java:
##
@@ -82,14 +83,17 @@ public Void visitPhysicalAggregate(PhysicalAggregate agg, PlanCo
addToRequestPropertyToChildren(PhysicalProperties.ANY);
return null;
}
-
+// EXCHANGE NODE does not need to be generated between DISTINCT_LOCAL
and GLOBAL
+if (agg.getAggPhase() == AggPhase.DISTINCT_LOCAL) {
+addToRequestPropertyToChildren(PhysicalProperties.ANY);
Review Comment:
we should set the request what the DISTINCT_LOCAL agg want to. in later
stage if the missing enforce helper found that the GLOBAL agg's distribute
satisfy the DISTINCT_LOCAL agg, no exchange node will be added
##
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/implementation/LogicalAggToPhysicalHashAgg.java:
##
@@ -36,6 +36,7 @@ public Rule build() {
ImmutableList.of(),
agg.getAggPhase(),
false,
+agg.isFinalPhase(),
Review Comment:
👍🏻
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangshuo128 commented on a diff in pull request #12461: [Feature](Nereids)add relation id
wangshuo128 commented on code in PR #12461:
URL: https://github.com/apache/doris/pull/12461#discussion_r965624981
##
fe/fe-core/src/main/java/org/apache/doris/nereids/StatementContext.java:
##
@@ -47,6 +51,10 @@ public StatementBase getParsedStatement() {
return parsedStatement;
}
+public RelationId getRelationId() {
Review Comment:
```suggestion
public RelationId nextRelationId() {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [enhancement](nereids) add syntax support for fractional literal (#12444)
This is an automated email from the ASF dual-hosted git repository.
morrysnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new d1ab6b1db2 [enhancement](nereids) add syntax support for fractional
literal (#12444)
d1ab6b1db2 is described below
commit d1ab6b1db285eeadfa4e779e9157dda56610cfcc
Author: Kikyou1997 <33112463+kikyou1...@users.noreply.github.com>
AuthorDate: Thu Sep 8 15:54:20 2022 +0800
[enhancement](nereids) add syntax support for fractional literal (#12444)
Just as legacy planner, Nereids parse all fractional literal to decimal.
In the future, we will add more syntax for user to control the fractional
literal type.
---
.../main/antlr4/org/apache/doris/nereids/DorisLexer.g4| 10 --
.../main/antlr4/org/apache/doris/nereids/DorisParser.g4 | 1 +
.../apache/doris/nereids/parser/LogicalPlanBuilder.java | 8
.../apache/doris/nereids/parser/NereidsParserTest.java| 15 +++
4 files changed, 24 insertions(+), 10 deletions(-)
diff --git a/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisLexer.g4
b/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisLexer.g4
index acdc465851..f076731864 100644
--- a/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisLexer.g4
+++ b/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisLexer.g4
@@ -432,16 +432,6 @@ DECIMAL_VALUE
: DECIMAL_DIGITS {isValidDecimal()}?
;
-FLOAT_LITERAL
-: DIGIT+ EXPONENT? 'F'
-| DECIMAL_DIGITS EXPONENT? 'F' {isValidDecimal()}?
-;
-
-DOUBLE_LITERAL
-: DIGIT+ EXPONENT? 'D'
-| DECIMAL_DIGITS EXPONENT? 'D' {isValidDecimal()}?
-;
-
BIGDECIMAL_LITERAL
: DIGIT+ EXPONENT? 'BD'
| DECIMAL_DIGITS EXPONENT? 'BD' {isValidDecimal()}?
diff --git a/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4
b/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4
index e1eba3ce6a..d2a03ecccf 100644
--- a/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4
+++ b/fe/fe-core/src/main/antlr4/org/apache/doris/nereids/DorisParser.g4
@@ -294,6 +294,7 @@ quotedIdentifier
number
: MINUS? INTEGER_VALUE#integerLiteral
+| MINUS? (EXPONENT_VALUE | DECIMAL_VALUE) #decimalLiteral
;
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
index 3f534c1c27..8874d0f047 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
@@ -27,6 +27,7 @@ import
org.apache.doris.nereids.DorisParser.ArithmeticUnaryContext;
import org.apache.doris.nereids.DorisParser.BooleanLiteralContext;
import org.apache.doris.nereids.DorisParser.ColumnReferenceContext;
import org.apache.doris.nereids.DorisParser.ComparisonContext;
+import org.apache.doris.nereids.DorisParser.DecimalLiteralContext;
import org.apache.doris.nereids.DorisParser.DereferenceContext;
import org.apache.doris.nereids.DorisParser.ExistContext;
import org.apache.doris.nereids.DorisParser.ExplainContext;
@@ -113,6 +114,7 @@ import
org.apache.doris.nereids.trees.expressions.literal.BigIntLiteral;
import org.apache.doris.nereids.trees.expressions.literal.BooleanLiteral;
import org.apache.doris.nereids.trees.expressions.literal.DateLiteral;
import org.apache.doris.nereids.trees.expressions.literal.DateTimeLiteral;
+import org.apache.doris.nereids.trees.expressions.literal.DecimalLiteral;
import org.apache.doris.nereids.trees.expressions.literal.IntegerLiteral;
import org.apache.doris.nereids.trees.expressions.literal.IntervalLiteral;
import org.apache.doris.nereids.trees.expressions.literal.LargeIntLiteral;
@@ -147,6 +149,7 @@ import org.antlr.v4.runtime.tree.ParseTree;
import org.antlr.v4.runtime.tree.RuleNode;
import org.antlr.v4.runtime.tree.TerminalNode;
+import java.math.BigDecimal;
import java.math.BigInteger;
import java.util.ArrayList;
import java.util.Arrays;
@@ -975,4 +978,9 @@ public class LogicalPlanBuilder extends
DorisParserBaseVisitor {
public List withInList(PredicateContext ctx) {
return
ctx.expression().stream().map(this::getExpression).collect(ImmutableList.toImmutableList());
}
+
+@Override
+public DecimalLiteral visitDecimalLiteral(DecimalLiteralContext ctx) {
+return new DecimalLiteral(new BigDecimal(ctx.getText()));
+}
}
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/nereids/parser/NereidsParserTest.java
b/fe/fe-core/src/test/java/org/apache/doris/nereids/parser/NereidsParserTest.java
index 208a45b09b..2fd3d757ac 100644
---
a/fe/fe-core/src/test/java/org/apache/doris/nereids/parser/NereidsParserTest.java
+++
b/fe/fe-core/src/test/java/org/apache/doris/nereids/parser/NereidsP
[GitHub] [doris] morrySnow merged pull request #12444: [enhancement](nereids) add syntax support for fractional literal
morrySnow merged PR #12444: URL: https://github.com/apache/doris/pull/12444 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangshuo128 commented on pull request #12461: [Feature](Nereids)add relation id
wangshuo128 commented on PR #12461: URL: https://github.com/apache/doris/pull/12461#issuecomment-1240361509 In general, I have a question. Why do we generate a relation id when binding relation instead of generation a unique relation id when creating `UnboundRelation` in the parser? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman closed issue #11699: [Bug] const fold not work for current_date when enable_fold_constant_by_be=false
morningman closed issue #11699: [Bug] const fold not work for current_date when enable_fold_constant_by_be=false URL: https://github.com/apache/doris/issues/11699 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman merged pull request #11702: [feature](function)Support function "current_date" in FE
morningman merged PR #11702: URL: https://github.com/apache/doris/pull/11702 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [feature](function)Support function "current_date" in FE (#11702)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 85bd29 [feature](function)Support function "current_date" in FE
(#11702)
85bd29 is described below
commit 85bd29b115a903932887676032d2f140dcdd
Author: qiye
AuthorDate: Thu Sep 8 16:00:57 2022 +0800
[feature](function)Support function "current_date" in FE (#11702)
Issue Number: close #11699
---
fe/fe-core/src/main/java/org/apache/doris/rewrite/FEFunctions.java | 5 +
.../src/test/java/org/apache/doris/planner/ConstantExpressTest.java | 4
2 files changed, 9 insertions(+)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/rewrite/FEFunctions.java
b/fe/fe-core/src/main/java/org/apache/doris/rewrite/FEFunctions.java
index cde27855cf..4cc39b5a18 100755
--- a/fe/fe-core/src/main/java/org/apache/doris/rewrite/FEFunctions.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/rewrite/FEFunctions.java
@@ -255,6 +255,11 @@ public class FEFunctions {
ScalarType.getDefaultDateType(Type.DATE));
}
+@FEFunction(name = "current_date", argTypes = {}, returnType = "DATE")
+public static DateLiteral currentDate() {
+return curDate();
+}
+
@FEFunction(name = "curtime", argTypes = {}, returnType = "TIME")
public static FloatLiteral curTime() throws AnalysisException {
DateLiteral now = now();
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/planner/ConstantExpressTest.java
b/fe/fe-core/src/test/java/org/apache/doris/planner/ConstantExpressTest.java
index d73a106ae6..e98d69647f 100644
--- a/fe/fe-core/src/test/java/org/apache/doris/planner/ConstantExpressTest.java
+++ b/fe/fe-core/src/test/java/org/apache/doris/planner/ConstantExpressTest.java
@@ -122,6 +122,10 @@ public class ConstantExpressTest {
testConstantExpressResult(
"select current_time();",
"");
+
+testConstantExpressResult(
+"select current_date();",
+"");
}
@Test
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] catpineapple opened a new pull request, #12473: [fix](doc) remove the ambiguity in the description of the operating instructions document
catpineapple opened a new pull request, #12473: URL: https://github.com/apache/doris/pull/12473 # Proposed changes Issue Number: close #xxx ## Problem summary Remove the ambiguity in the description of the operating instructions document ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris-website] catpineapple opened a new pull request, #89: [fix](doc) remove the ambiguity in the description of the operating instructions document
catpineapple opened a new pull request, #89: URL: https://github.com/apache/doris-website/pull/89 Remove the ambiguity in the description of the operating instructions document -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] carlvinhust2012 opened a new pull request, #12474: [regression](array-type) add some case for array insert
carlvinhust2012 opened a new pull request, #12474: URL: https://github.com/apache/doris/pull/12474 # Proposed changes 1.this pr is used to add some case for array insert. Issue Number: #7570 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #12081: [Fix](BE) update pid file after be launch succeed
hello-stephen commented on PR #12081:
URL: https://github.com/apache/doris/pull/12081#issuecomment-1240401634
when BE is already started, then run "sh start_be.sh --daemon", be.pid file
will be overwritten, Do you mean this?
At present, start_be.sh will check if PID process already exists and throw
error info if it does.
In some situations, the PID in be.pid file is not the actual PID of
doris_be, and if run start_be.sh, the be.pid file will indeed be overwritten，
for example， if run this `for i in {1..10};do bash start_be.sh --daemon &
done`, the pid in be.pid is probably wrong，and if run start_be.sh again, the
be.pid file will indeed be overwritten.
So, it looks like this pr didn't really fix the problem.
any other idea?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly opened a new pull request, #12475: [fix](nereids) there are some invalid slot in other join conjuncts (hash join)
englefly opened a new pull request, #12475: URL: https://github.com/apache/doris/pull/12475 # Proposed changes test sql: tpch q21 ``` select count(*) from lineitem l3 right anti join lineitem l1 on l3.l_orderkey = l1.l_orderkey and l3.l_suppkey <> l1.l_suppkey; ``` if we have other join conjuncts, we have to put all slots from left and right into `slotReferenceMap` instead of `hashjoin.getOutput()` Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yangzhg merged pull request #12460: [fix](stream load) Fix wrong conversion of null value when vstream load json format
yangzhg merged PR #12460: URL: https://github.com/apache/doris/pull/12460 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](stream load) Fix wrong conversion of null value when vstream load json format (#12460)
This is an automated email from the ASF dual-hosted git repository.
yangzhg pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 2ccbbb5392 [fix](stream load) Fix wrong conversion of null value when
vstream load json format (#12460)
2ccbbb5392 is described below
commit 2ccbbb53922016edbd491f63643454689cbe24f9
Author: yinzhijian <373141...@qq.com>
AuthorDate: Thu Sep 8 16:48:35 2022 +0800
[fix](stream load) Fix wrong conversion of null value when vstream load
json format (#12460)
---
be/src/vec/exec/vjson_scanner.cpp | 13 -
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/be/src/vec/exec/vjson_scanner.cpp
b/be/src/vec/exec/vjson_scanner.cpp
index dccbdd5241..dbe6d986e4 100644
--- a/be/src/vec/exec/vjson_scanner.cpp
+++ b/be/src/vec/exec/vjson_scanner.cpp
@@ -294,7 +294,12 @@ Status
VJsonReader::_write_data_to_column(rapidjson::Value::ConstValueIterator v
vectorized::ColumnNullable* nullable_column = nullptr;
if (slot_desc->is_nullable()) {
nullable_column =
reinterpret_cast(column_ptr);
-nullable_column->get_null_map_data().push_back(0);
+// kNullType will put 1 into the Null map, so there is no need to push
0 for kNullType.
+if (value->GetType() != rapidjson::Type::kNullType) {
+nullable_column->get_null_map_data().push_back(0);
+} else {
+nullable_column->insert_default();
+}
column_ptr = &nullable_column->get_nested_column();
}
@@ -326,9 +331,7 @@ Status
VJsonReader::_write_data_to_column(rapidjson::Value::ConstValueIterator v
str_value = (char*)"1";
break;
case rapidjson::Type::kNullType:
-if (slot_desc->is_nullable()) {
-nullable_column->insert_default();
-} else {
+if (!slot_desc->is_nullable()) {
RETURN_IF_ERROR(_append_error_msg(
*value, "Json value is null, but the column `{}` is not
nullable.",
slot_desc->col_name(), valid));
@@ -630,7 +633,7 @@ Status
VSIMDJsonReader::_write_data_to_column(simdjson::ondemand::value value,
ColumnString* column_string = assert_cast(column_ptr);
if (value.is_null()) {
if (column->is_nullable()) {
-nullable_column->get_null_map_data().push_back(1);
+// insert_default already push 1 to null_map
nullable_column->insert_default();
} else {
RETURN_IF_ERROR(
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #12468: [feature-wip](nereids) implement uncheckedCast method in VarcharLiteral
morrySnow commented on PR #12468: URL: https://github.com/apache/doris/pull/12468#issuecomment-1240424043 please add regression test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhengshiJ commented on a diff in pull request #12421: [fix](nereids) fix uncorrelated subquery can't get the correct result
zhengshiJ commented on code in PR #12421:
URL: https://github.com/apache/doris/pull/12421#discussion_r965687294
##
fe/fe-core/src/main/java/org/apache/doris/planner/CrossJoinNode.java:
##
@@ -57,6 +58,26 @@ public CrossJoinNode(PlanNodeId id, PlanNode outer, PlanNode
inner, TableRef inn
nullableTupleIds.addAll(inner.getNullableTupleIds());
}
+/**
+ * .
Review Comment:
done
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalAssertNumRows.java:
##
@@ -59,7 +60,8 @@ public AssertNumRowsElement getAssertNumRowsElement() {
@Override
public String toString() {
-return "LogicalAssertNumRows (" + assertNumRowsElement + ")";
+return Utils.toSqlString("LogicalAssertNumRows (",
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #12468: [feature-wip](nereids) implement uncheckedCast method in VarcharLiteral
morrySnow commented on code in PR #12468:
URL: https://github.com/apache/doris/pull/12468#discussion_r965687421
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/literal/VarcharLiteral.java:
##
@@ -63,4 +67,28 @@ public LiteralExpr toLegacyLiteral() {
public String toString() {
return "'" + value + "'";
}
+
+@Override
+protected Expression uncheckedCastTo(DataType targetType) throws
AnalysisException {
+if (getDataType().equals(targetType)) {
+return this;
+}
+if (targetType.isDateType()) {
+return convertToDate(targetType);
+} else if (targetType.isIntType()) {
+return new IntegerLiteral(Integer.parseInt(value));
+}
+//todo other target type cast
Review Comment:
remove this TODO, and add a comment to this function: temporary way to
process type coercion in TimestampArithmetic, should be replaced by
TypeCoercion rule.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhengshiJ commented on a diff in pull request #12421: [fix](nereids) fix uncorrelated subquery can't get the correct result
zhengshiJ commented on code in PR #12421:
URL: https://github.com/apache/doris/pull/12421#discussion_r965686990
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/AssertNumRowsElement.java:
##
@@ -75,7 +75,7 @@ public AssertNumRowsElement withChildren(List
children) {
public String toString() {
return Utils.toSqlString("desiredNumOfRows: ",
Long.toString(desiredNumOfRows),
-"assertion: " + assertion);
+"assertion: ", assertion);
Review Comment:
done
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalAssertNumRows.java:
##
@@ -59,7 +60,8 @@ public AssertNumRowsElement getAssertNumRowsElement() {
@Override
public String toString() {
-return "LogicalAssertNumRows (" + assertNumRowsElement + ")";
+return Utils.toSqlString("LogicalAssertNumRows (",
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee commented on a diff in pull request #12349: [Improvement](sort) improve partial sort algorithm
HappenLee commented on code in PR #12349:
URL: https://github.com/apache/doris/pull/12349#discussion_r965668178
##
be/src/vec/core/sort_block.h:
##
@@ -46,9 +48,425 @@ void stable_get_permutation(const Block& block, const
SortDescription& descripti
*/
bool is_already_sorted(const Block& block, const SortDescription& description);
-using ColumnsWithSortDescriptions = std::vector>;
+using ColumnWithSortDescription = std::pair;
+
+using ColumnsWithSortDescriptions = std::vector;
ColumnsWithSortDescriptions get_columns_with_sort_description(const Block&
block,
const
SortDescription& description);
+struct EqualRangeIterator {
+int range_begin;
+int range_end;
+
+EqualRangeIterator(const EqualFlags& flags) : EqualRangeIterator(flags, 0,
flags.size()) {}
+
+EqualRangeIterator(const EqualFlags& flags, int begin, int end) :
flags_(flags), end_(end) {
+range_begin = begin;
+range_end = end;
+cur_range_begin_ = begin;
+cur_range_end_ = end;
+}
+
+bool next() {
+if (cur_range_begin_ >= end_) {
+return false;
+}
+
+// `flags_[i]=1` indicates that the i-th row is equal to the previous
row, which means we
+// should continue to sort this row according to current column. Using
the first non-zero
+// value and first zero value after first non-zero value as two
bounds, we can get an equal range here
+if (!(cur_range_begin_ == 0) || !(flags_[cur_range_begin_] == 1)) {
+cur_range_begin_ = simd::find_nonzero(flags_, cur_range_begin_ +
1);
+if (cur_range_begin_ >= end_) {
+return false;
+}
+cur_range_begin_--;
+}
+
+cur_range_end_ = simd::find_zero(flags_, cur_range_begin_ + 1);
+cur_range_end_ = std::min(cur_range_end_, end_);
Review Comment:
which case `cur_range_end` smaller than `end_`
##
be/src/vec/core/sort_block.h:
##
@@ -46,9 +48,425 @@ void stable_get_permutation(const Block& block, const
SortDescription& descripti
*/
bool is_already_sorted(const Block& block, const SortDescription& description);
-using ColumnsWithSortDescriptions = std::vector>;
+using ColumnWithSortDescription = std::pair;
+
+using ColumnsWithSortDescriptions = std::vector;
ColumnsWithSortDescriptions get_columns_with_sort_description(const Block&
block,
const
SortDescription& description);
+struct EqualRangeIterator {
+int range_begin;
+int range_end;
+
+EqualRangeIterator(const EqualFlags& flags) : EqualRangeIterator(flags, 0,
flags.size()) {}
+
+EqualRangeIterator(const EqualFlags& flags, int begin, int end) :
flags_(flags), end_(end) {
+range_begin = begin;
+range_end = end;
+cur_range_begin_ = begin;
+cur_range_end_ = end;
+}
+
+bool next() {
+if (cur_range_begin_ >= end_) {
+return false;
+}
+
+// `flags_[i]=1` indicates that the i-th row is equal to the previous
row, which means we
+// should continue to sort this row according to current column. Using
the first non-zero
+// value and first zero value after first non-zero value as two
bounds, we can get an equal range here
+if (!(cur_range_begin_ == 0) || !(flags_[cur_range_begin_] == 1)) {
+cur_range_begin_ = simd::find_nonzero(flags_, cur_range_begin_ +
1);
+if (cur_range_begin_ >= end_) {
+return false;
+}
+cur_range_begin_--;
+}
+
+cur_range_end_ = simd::find_zero(flags_, cur_range_begin_ + 1);
+cur_range_end_ = std::min(cur_range_end_, end_);
+
+if (cur_range_begin_ >= cur_range_end_) {
+return false;
+}
+
+range_begin = cur_range_begin_;
+range_end = cur_range_end_;
+cur_range_begin_ = cur_range_end_;
+return true;
+}
+
+private:
+int cur_range_begin_;
+int cur_range_end_;
+
+const EqualFlags& flags_;
+const int end_;
+};
+
+struct ColumnPartialSortingLess {
+const ColumnWithSortDescription& column_;
+
+explicit ColumnPartialSortingLess(const ColumnWithSortDescription& column)
: column_(column) {}
+
+bool operator()(size_t a, size_t b) const {
+int res = column_.second.direction *
+ column_.first->compare_at(a, b, *column_.first,
column_.second.nulls_direction);
+if (res < 0) {
+return true;
+} else if (res > 0) {
+return false;
+}
+return false;
+}
+};
+
+template
+struct PermutationWithInlineValue {
+T inline_value_;
+uint32_t row_id_;
+
+PermutationWithInlineValue(T inline_value, uint32_t row_id)
+: inline_value_(inline_value), row_id_(row_id) {}
+
+PermutationWithInlineValue
[GitHub] [doris] gongzexin opened a new issue, #12476: [Bug] The execution plan is partitoned, not bucket shuffle join
gongzexin opened a new issue, #12476: URL: https://github.com/apache/doris/issues/12476 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version 1.1.1-rc03 ### What's Wrong? The enable_bucket_shuffle_join variable is set to true and the join hint is used, but the explain is partitioned ### What You Expected? bucket shuffle join ### How to Reproduce? Table schema and data refer to [TPC-H](https://doris.apache.org/zh-CN/docs/benchmark/tpch), query sql: `explain select * from nation join[shuffle] region on n_nationkey = r_regionkey` ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] AshinGau opened a new pull request, #12477: [feature-wip](parquet-reader) bug fix, parquet footer buffer is small when containing many columns
AshinGau opened a new pull request, #12477: URL: https://github.com/apache/doris/pull/12477 # Proposed changes ## Problem summary Failed when reading parquet file with many columns(>1600). ``` mysql> select int_col from types_sf100_r100w limit 5; ERROR 1105 (HY000): errCode = 2, detailMessage = Couldn't deserialize thrift msg: TProtocolException: Invalid data ``` ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [x] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [x] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on a diff in pull request #12316: [Enhancement](compaction) reduce VMergeIterator copy block
BiteThet commented on code in PR #12316:
URL: https://github.com/apache/doris/pull/12316#discussion_r965712388
##
be/src/olap/rowset/beta_rowset_reader.cpp:
##
@@ -313,6 +314,27 @@ Status BetaRowsetReader::next_block(vectorized::Block*
block) {
return Status::OK();
}
+Status BetaRowsetReader::next_block_view(vectorized::BlockView* block_view) {
+SCOPED_RAW_TIMER(&_stats->block_fetch_ns);
+if (config::enable_storage_vectorization && _context->is_vec) {
+do {
+auto s = _iterator->next_block_view(block_view);
+if (!s.ok()) {
+if (s.is_end_of_file()) {
+return Status::OLAPInternalError(OLAP_ERR_DATA_EOF);
+} else {
+LOG(WARNING) << "failed to read next block: " <<
s.to_string();
+return
Status::OLAPInternalError(OLAP_ERR_ROWSET_READ_FAILED);
+}
+}
+} while (block_view->empty());
+} else {
+return Status::NotSupported("block view only support
enable_storage_vectorization");
Review Comment:
use ```cpp get_data_by_ref = get_data_by_ref &&
_rs_reader->support_return_data_by_ref() &&
config::enable_storage_vectorization``` to control it
now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon commented on a diff in pull request #12293: [Enhancement](array-type) record offsets info to speed up array colum…
eldenmoon commented on code in PR #12293: URL: https://github.com/apache/doris/pull/12293#discussion_r965716438 ## be/src/common/config.h: ## @@ -849,6 +849,8 @@ CONF_Bool(enable_simdjson_reader, "false"); CONF_Int32(doris_remote_scanner_thread_pool_thread_num, "16"); // number of s3 scanner thread pool queue size CONF_Int32(doris_remote_scanner_thread_pool_queue_size, "10240"); +// write offsets instead of length to speed up seek performance +CONF_Bool(enable_write_array_offset, "false"); Review Comment: I had a conversation with @cambyzju and decided to use `offset version`, so this PR is incompatible with previous -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris-website] hf200012 merged pull request #90: fix PWA register error
hf200012 merged PR #90: URL: https://github.com/apache/doris-website/pull/90 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix PWA register error (#90)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 5742bb5a42e fix PWA register error (#90)
5742bb5a42e is described below
commit 5742bb5a42ed188d4c3777acc2e60e4f849a3003
Author: song7788q
AuthorDate: Thu Sep 8 17:23:06 2022 +0800
fix PWA register error (#90)
---
docusaurus.config.js | 4 +++-
package.json | 2 +-
static/manifest.json | 2 +-
3 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/docusaurus.config.js b/docusaurus.config.js
index e7a92ca94b9..383ca8d7ce6 100644
--- a/docusaurus.config.js
+++ b/docusaurus.config.js
@@ -7,6 +7,8 @@ const showAllVersions = true;
const { ssrTemplate } = require('./config/ssrTemplate');
const customDocusaurusPlugin = require('./config/custom-docusaurus-plugin');
+console.log(process.env);
+
/** @type {import('@docusaurus/types').Config} */
const config = {
title: 'Apache Doris',
@@ -52,7 +54,7 @@ const config = {
[
'@docusaurus/plugin-pwa',
{
-debug: false,
+debug: true,
offlineModeActivationStrategies: ['appInstalled',
'standalone', 'queryString', 'mobile'],
injectManifestConfig: {
globPatterns:
['**/*.{json,pdf,docx,xlsx,html,css,js,png,svg,ico,jpg,jpeg}'],
diff --git a/package.json b/package.json
index 6b8aa194fb6..96868b0556f 100644
--- a/package.json
+++ b/package.json
@@ -6,7 +6,7 @@
"docusaurus": "docusaurus",
"start": "docusaurus start",
"start:zh-CN": "docusaurus start --locale zh-CN",
-"build": "docusaurus build",
+"build": "PWA_SERVICE_WORKER_URL=https://doris.apache.org docusaurus
build",
"swizzle": "docusaurus swizzle",
"deploy": "docusaurus deploy",
"clear": "docusaurus clear",
diff --git a/static/manifest.json b/static/manifest.json
index 066003cc009..1f973a19551 100644
--- a/static/manifest.json
+++ b/static/manifest.json
@@ -5,7 +5,7 @@
"background_color": "#FF",
"display": "standalone",
"scope": "/",
-"start_url": "https://doris.apache.org";,
+"start_url": "./index.html",
"related_applications": [
{
"platform": "webapp",
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris-website] hf200012 merged pull request #89: [fix](doc) remove the ambiguity in the description of the operating instructions document
hf200012 merged PR #89: URL: https://github.com/apache/doris-website/pull/89 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon commented on a diff in pull request #12293: [Enhancement](array-type) record offsets info to speed up array colum…
eldenmoon commented on code in PR #12293: URL: https://github.com/apache/doris/pull/12293#discussion_r965716438 ## be/src/common/config.h: ## @@ -849,6 +849,8 @@ CONF_Bool(enable_simdjson_reader, "false"); CONF_Int32(doris_remote_scanner_thread_pool_thread_num, "16"); // number of s3 scanner thread pool queue size CONF_Int32(doris_remote_scanner_thread_pool_queue_size, "10240"); +// write offsets instead of length to speed up seek performance +CONF_Bool(enable_write_array_offset, "false"); Review Comment: I had a conversation with @cambyzju and decided to use `offset version`, so this PR is incompatible with previous. new Performace Test added to https://github.com/apache/doris/issues/12246 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: delete_doc_upd (#89)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new c2989c7ca59 delete_doc_upd (#89)
c2989c7ca59 is described below
commit c2989c7ca594ae3ba468494d6fd98d9b5e3cbd12
Author: catpineapple <42031973+catpineap...@users.noreply.github.com>
AuthorDate: Thu Sep 8 17:23:24 2022 +0800
delete_doc_upd (#89)
---
.../Data-Manipulation-Statements/Manipulation/DELETE.md | 2 +-
.../Data-Manipulation-Statements/Manipulation/DELETE.md | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git
a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/DELETE.md
b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/DELETE.md
index 447a64cc789..d1da26c111a 100644
---
a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/DELETE.md
+++
b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/DELETE.md
@@ -47,7 +47,7 @@ column_name1 op { value | value_list } [ AND column_name2 op
{ value | value_lis
illustrate:
1. The optional types of op include: =, >, <, >=, <=, !=, in, not in
-2. Only conditions on the key column can be specified.
+2. Only conditions on the key column can be specified when using AGGREGATE
(UNIQUE) model.
3. When the selected key column does not exist in a rollup, delete cannot be
performed.
4. Conditions can only have an "and" relationship. If you want to achieve an
"or" relationship, you need to write the conditions in two DELETE statements.
5. If it is a partitioned table, you can specify a partition. If not
specified, and the session variable delete_without_partition is true, it will
be applied to all partitions. If it is a single-partition table, it can be left
unspecified.
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/DELETE.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/DELETE.md
index 5a7f0417fb2..24daa0147c1 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/DELETE.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-reference/Data-Manipulation-Statements/Manipulation/DELETE.md
@@ -46,9 +46,9 @@ column_name1 op { value | value_list } [ AND column_name2 op
{ value | value_lis
说明：
1. op 的可选类型包括：=, >, <, >=, <=, !=, in, not in
-2. 只能指定 key 列上的条件。
-3. 当选定的 key 列不存在于某个 rollup 中时，无法进行 delete。
-4. 条件之间只能是“与”的关系。若希望达成“或”的关系，需要将条件分写在两个 DELETE 语句中。
+2. 使用聚合类的表模型（AGGREGATE、UNIQUE）只能指定 key 列上的条件。
+3. 当选定的 key 列不存在于某个 rollup 中时，无法进行 delete。
+4. 条件之间只能是“与”的关系。若希望达成“或”的关系，需要将条件分写在两个 DELETE 语句中。
5. 如果为分区表，可以指定分区，如不指定，且会话变量 delete_without_partition 为
true，则会应用到所有分区。如果是单分区表，可以不指定。
注意：
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris-website] hf200012 merged pull request #87: hide learning path in community
hf200012 merged PR #87: URL: https://github.com/apache/doris-website/pull/87 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: hide learning path in community (#87)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new f503a79a36a hide learning path in community (#87)
f503a79a36a is described below
commit f503a79a36a0f65736b9e91a55ee9f92faf8d9e6
Author: wangyongfeng <943155...@qq.com>
AuthorDate: Thu Sep 8 17:23:57 2022 +0800
hide learning path in community (#87)
hide learning path in community
---
src/theme/DocSidebar/Desktop/index.tsx | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/src/theme/DocSidebar/Desktop/index.tsx
b/src/theme/DocSidebar/Desktop/index.tsx
index 37045a48d26..42e78ae72e4 100644
--- a/src/theme/DocSidebar/Desktop/index.tsx
+++ b/src/theme/DocSidebar/Desktop/index.tsx
@@ -15,17 +15,18 @@ function DocSidebarDesktop({ path, sidebar, onCollapse,
isHidden }) {
sidebar: { hideable },
},
} = useThemeConfig();
+const hasLearningPath = path.includes('/docs')
return (
{hideOnScroll && }
-
+{hasLearningPath &&
Learning Path
-
+}
{hideable && }
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xy720 opened a new pull request, #12479: [Bug](array_type) Forbid adding array key columns
xy720 opened a new pull request, #12479: URL: https://github.com/apache/doris/pull/12479 # Proposed changes ``` mysql> desc array_test; +---++--+---+-+---+ | Field | Type | Null | Key | Default | Extra | +---++--+---+-+---+ | id| INT| Yes | true | NULL| | | c_array | ARRAY | Yes | false | NULL| NONE | +---++--+---+-+---+ Before: mysql> ALTER TABLE array_test ADD COLUMN add_arr_key array key NULL DEFAULT NULL; Query OK, 0 rows affected (0.00 sec) After: mysql> ALTER TABLE array_test MODIFY COLUMN c_array array key NULL DEFAULT NULL; ERROR 1105 (HY000): errCode = 2, detailMessage = Array can only be used in the non-key column of the duplicate table at present. ``` ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee commented on a diff in pull request #12349: [Improvement](sort) improve partial sort algorithm
HappenLee commented on code in PR #12349:
URL: https://github.com/apache/doris/pull/12349#discussion_r965753805
##
be/src/vec/core/sort_block.h:
##
@@ -46,9 +48,425 @@ void stable_get_permutation(const Block& block, const
SortDescription& descripti
*/
bool is_already_sorted(const Block& block, const SortDescription& description);
-using ColumnsWithSortDescriptions = std::vector>;
+using ColumnWithSortDescription = std::pair;
+
+using ColumnsWithSortDescriptions = std::vector;
ColumnsWithSortDescriptions get_columns_with_sort_description(const Block&
block,
const
SortDescription& description);
+struct EqualRangeIterator {
+int range_begin;
+int range_end;
+
+EqualRangeIterator(const EqualFlags& flags) : EqualRangeIterator(flags, 0,
flags.size()) {}
+
+EqualRangeIterator(const EqualFlags& flags, int begin, int end) :
flags_(flags), end_(end) {
+range_begin = begin;
+range_end = end;
+cur_range_begin_ = begin;
+cur_range_end_ = end;
+}
+
+bool next() {
+if (cur_range_begin_ >= end_) {
+return false;
+}
+
+// `flags_[i]=1` indicates that the i-th row is equal to the previous
row, which means we
+// should continue to sort this row according to current column. Using
the first non-zero
+// value and first zero value after first non-zero value as two
bounds, we can get an equal range here
+if (!(cur_range_begin_ == 0) || !(flags_[cur_range_begin_] == 1)) {
+cur_range_begin_ = simd::find_nonzero(flags_, cur_range_begin_ +
1);
+if (cur_range_begin_ >= end_) {
+return false;
+}
+cur_range_begin_--;
+}
+
+cur_range_end_ = simd::find_zero(flags_, cur_range_begin_ + 1);
+cur_range_end_ = std::min(cur_range_end_, end_);
+
+if (cur_range_begin_ >= cur_range_end_) {
+return false;
+}
+
+range_begin = cur_range_begin_;
+range_end = cur_range_end_;
+cur_range_begin_ = cur_range_end_;
+return true;
+}
+
+private:
+int cur_range_begin_;
+int cur_range_end_;
+
+const EqualFlags& flags_;
+const int end_;
+};
+
+struct ColumnPartialSortingLess {
+const ColumnWithSortDescription& column_;
+
+explicit ColumnPartialSortingLess(const ColumnWithSortDescription& column)
: column_(column) {}
+
+bool operator()(size_t a, size_t b) const {
+int res = column_.second.direction *
+ column_.first->compare_at(a, b, *column_.first,
column_.second.nulls_direction);
+if (res < 0) {
+return true;
+} else if (res > 0) {
+return false;
+}
+return false;
+}
+};
+
+template
+struct PermutationWithInlineValue {
+T inline_value_;
+uint32_t row_id_;
+
+PermutationWithInlineValue(T inline_value, uint32_t row_id)
+: inline_value_(inline_value), row_id_(row_id) {}
+
+PermutationWithInlineValue() : row_id_(-1) {}
+};
+
+template
+using PermutationForColumn = std::vector>;
+
+class ColumnSorter {
+public:
+explicit ColumnSorter(const ColumnWithSortDescription& column, const int
limit)
+: column_(column),
+ limit_(limit),
+ nulls_direction_(column.second.nulls_direction),
+ direction_(column.second.direction) {}
+
+void operator()(EqualFlags& flags, IColumn::Permutation& perms,
EqualRange& range,
+bool last_column) const {
+column_.first->sort_column(this, flags, perms, range, last_column);
+}
+
+void sort_column(const IColumn& column, EqualFlags& flags,
IColumn::Permutation& perms,
+ EqualRange& range, bool last_column) const {
+int new_limit = limit_;
+auto comparator = [&](const size_t a, const size_t b) {
+return column.compare_at(a, b, *column_.first, nulls_direction_);
+};
+ColumnPartialSortingLess less(column_);
+auto do_sort = [&](size_t first_iter, size_t last_iter) {
+auto begin = perms.begin() + first_iter;
+auto end = perms.begin() + last_iter;
+
+if (UNLIKELY(limit_ > 0 && first_iter < limit_ && limit_ <=
last_iter)) {
+int n = limit_ - first_iter;
+std::partial_sort(begin, begin + n, end, less);
+
+auto nth = perms[limit_ - 1];
+size_t equal_count = 0;
+for (auto iter = begin + n; iter < end; iter++) {
+if (comparator(*iter, nth) == 0) {
+std::iter_swap(iter, begin + n + equal_count);
+equal_count++;
+}
+}
+new_limit = limit_ + equal_count;
+} else {
+pdqsort(begin, end, less);
+}
+};
+
+
[GitHub] [doris] eldenmoon commented on pull request #12479: [Bug](array_type) Forbid adding array key columns
eldenmoon commented on PR #12479: URL: https://github.com/apache/doris/pull/12479#issuecomment-1240498900 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] FreeOnePlus closed pull request #12440: [Feature-wip](docker)Initialize and submit code for rapid development of Docker
FreeOnePlus closed pull request #12440: [Feature-wip](docker)Initialize and submit code for rapid development of Docker URL: https://github.com/apache/doris/pull/12440 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] FreeOnePlus opened a new pull request, #12480: [Feature-wip](docker)Initialize and submit code for rapid development of Docker
FreeOnePlus opened a new pull request, #12480: URL: https://github.com/apache/doris/pull/12480 # Proposed changes Submitted shell script code for Dockerfile, Docker-Compose and related registers. ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] 924060929 opened a new pull request, #12481: (feature)(Nereids) Support function registry
924060929 opened a new pull request, #12481:
URL: https://github.com/apache/doris/pull/12481
# Proposed changes
Support function registry.
The classes:
- FunctionRegistry: used to register scalar functions and aggregate functions
- FunctionBuilder: used to resolve a BoundFunction, extract the constructor,
and build to a BoundFunction by arguments(`List`)
Register example:
```java
public class FunctionRegistry {
public static final List SCALAR_FUNCTIONS = ImmutableList.of(
scalar(Substring.class, "substr", "substring"),
scalar(WeekOfYear.class),
scalar(Year.class)
);
public static final ImmutableList AGGREGATE_FUNCTIONS =
ImmutableList.of(
agg(Avg.class),
agg(Count.class),
agg(Max.class),
agg(Min.class),
agg(Sum.class)
);
}
```
Note:
- Currently, we only support register scalar functions add aggregate
functions, we will support register table functions.
- Currently, we only support resolve function by function name and
difference arity, but can not resolve the same arity override function, e.g.
`some_function(Expression)` and `some_function(Literal)`
## Checklist(Required)
1. Does it affect the original behavior:
- [ ] Yes
- [x] No
- [ ] I don't know
2. Has unit tests been added:
- [x] Yes
- [ ] No
- [ ] No Need
3. Has document been added or modified:
- [ ] Yes
- [ ] No
- [x] No Need
4. Does it need to update dependencies:
- [ ] Yes
- [x] No
5. Are there any changes that cannot be rolled back:
- [ ] Yes (If Yes, please explain WHY)
- [x] No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangbo commented on issue #12466: [Bug] 1.1 version Column prune incorrect result
wangbo commented on issue #12466: URL: https://github.com/apache/doris/issues/12466#issuecomment-1240557989 ### Solution 1 generate a _column_to_check for HashJoinNode, by using HashJoinNode's _row_desc_for_other_join_conjunt or _row_descriptor. 2 We fill column by -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangbo closed issue #12466: [Bug] 1.1 version Column prune incorrect result
wangbo closed issue #12466: [Bug] 1.1 version Column prune incorrect result URL: https://github.com/apache/doris/issues/12466 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] freesinger commented on pull request #10322: [feature](JSON datatype)Support JSON datatype
freesinger commented on PR #10322: URL: https://github.com/apache/doris/pull/10322#issuecomment-1240574927 Sample records are as follows:  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 opened a new pull request, #12482: [feature-wip](nereids) Cast children type of cmp expression to the common type
Kikyou1997 opened a new pull request, #12482: URL: https://github.com/apache/doris/pull/12482 # Proposed changes Issue Number: noissue ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #12447: [enhancement](Nereids)change aggregate and join stats calc algorithm
github-actions[bot] commented on PR #12447: URL: https://github.com/apache/doris/pull/12447#issuecomment-1240580682 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #12447: [enhancement](Nereids)change aggregate and join stats calc algorithm
github-actions[bot] commented on PR #12447: URL: https://github.com/apache/doris/pull/12447#issuecomment-1240580733 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on pull request #12482: [feature-wip](nereids) Cast children type of cmp expression to the common type
Kikyou1997 commented on PR #12482: URL: https://github.com/apache/doris/pull/12482#issuecomment-1240591267 @morrySnow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #12293: [Enhancement](array-type) record offsets info to speed up array colum…
cambyzju commented on code in PR #12293:
URL: https://github.com/apache/doris/pull/12293#discussion_r965844526
##
be/src/olap/column_vector.h:
##
@@ -233,6 +233,13 @@ class ArrayColumnVectorBatch : public ColumnVectorBatch {
*/
void get_offset_by_length(size_t start_idx, size_t size);
+// From `start_idx`, put `size` ordinals to _item_offsets
+// Ex:
+// original _item_offsets: 0 3 5 9; ordinals to be added: 100 105 111;
size: 3; satart_idx: 3
+// --> _item_offsets: 0 3 5 (9 + 100 - 100) (9 + 105 - 100) (9 + 111 - 100)
+// _item_offsets becomes 0 3 5 9 14 20
+void put_item_ordinal(segment_v2::ordinal_t* ordinals, size_t start_idx,
size_t size);
Review Comment:
does push back is enough? it seems we always append ordinals at the end.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow opened a new pull request, #12483: [enhancement](Nereids) add all necessary PhysicalDistribute in CostAndEnforcerJob
morrySnow opened a new pull request, #12483: URL: https://github.com/apache/doris/pull/12483 # Proposed changes In an earlier PR #11976 , we add shuffle join and bucket shuffle support. But if join's right child's distribution spec satisfied join's require, we do not add distribute on right child. Instead of, do it in plan translator. It is hard to calculate accurate cost in this way, since we some distribute cost do not calculated. In this PR, we introduce a new shuffle type BUCKET, and change the way of add enforce to ensure all necessary distribute will be added in cost and enforcer job. ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] freemandealer opened a new issue, #12484: [Bug](vectorized load) incomplete errmsg when find partition failed
freemandealer opened a new issue, #12484: URL: https://github.com/apache/doris/issues/12484 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version updated master branch ### What's Wrong? error msg incomplete when find partition failed  ### What You Expected? detailed and useful info for bad tuples ### How to Reproduce? 1. create a table with partitions that cannot cover the whole range of data 2. load data (contains row that out of range so that partition will not be found for it) ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] freemandealer opened a new pull request, #12485: [fix](vectorized load) fix incomplete errmsg when find partition failed [#12484]
freemandealer opened a new pull request, #12485: URL: https://github.com/apache/doris/pull/12485 Signed-off-by: freemandealer # Proposed changes Issue Number: close #12484 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [] Yes - [X] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [X] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [X] No Need 4. Does it need to update dependencies: - [ ] Yes - [X] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [X] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon commented on a diff in pull request #12293: [Enhancement](array-type) record offsets info to speed up array colum…
eldenmoon commented on code in PR #12293:
URL: https://github.com/apache/doris/pull/12293#discussion_r965854049
##
be/src/olap/column_vector.h:
##
@@ -233,6 +233,13 @@ class ArrayColumnVectorBatch : public ColumnVectorBatch {
*/
void get_offset_by_length(size_t start_idx, size_t size);
+// From `start_idx`, put `size` ordinals to _item_offsets
+// Ex:
+// original _item_offsets: 0 3 5 9; ordinals to be added: 100 105 111;
size: 3; satart_idx: 3
+// --> _item_offsets: 0 3 5 (9 + 100 - 100) (9 + 105 - 100) (9 + 111 - 100)
+// _item_offsets becomes 0 3 5 9 14 20
+void put_item_ordinal(segment_v2::ordinal_t* ordinals, size_t start_idx,
size_t size);
Review Comment:
yes, it's enough we just need to caculate the ordinals read from
_offset_iterator, and then put them into the back of ArrayColumnVectorBatch's
_offsets.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #12293: [Enhancement](array-type) record offsets info to speed up array colum…
cambyzju commented on code in PR #12293: URL: https://github.com/apache/doris/pull/12293#discussion_r965855818 ## be/src/olap/rowset/segment_v2/column_reader.cpp: ## @@ -410,18 +410,33 @@ Status ArrayFileColumnIterator::next_batch(size_t* n, ColumnBlockView* dst, bool ColumnBlock* array_block = dst->column_block(); auto* array_batch = static_cast(array_block->vector_batch()); -// 1. read n offsets +// 1. read n+1 offsets +array_batch->offsets()->resize(*n + 1); ColumnBlock offset_block(array_batch->offsets(), nullptr); -ColumnBlockView offset_view(&offset_block, -dst->current_offset() + 1); // offset应该比collection的游标多1 +ColumnBlockView offset_view(&offset_block); Review Comment: first element of offsets is always zero, should we still need to read? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #12293: [Enhancement](array-type) record offsets info to speed up array colum…
cambyzju commented on code in PR #12293:
URL: https://github.com/apache/doris/pull/12293#discussion_r965858876
##
be/src/olap/column_vector.h:
##
@@ -233,6 +233,13 @@ class ArrayColumnVectorBatch : public ColumnVectorBatch {
*/
void get_offset_by_length(size_t start_idx, size_t size);
+// From `start_idx`, put `size` ordinals to _item_offsets
+// Ex:
+// original _item_offsets: 0 3 5 9; ordinals to be added: 100 105 111;
size: 3; satart_idx: 3
+// --> _item_offsets: 0 3 5 (9 + 100 - 100) (9 + 105 - 100) (9 + 111 - 100)
Review Comment:
actually we only append `size - 1` offsets:
```suggestion
// --> _item_offsets: 0 3 5 9 (9 + 105 - 100) (9 + 111 - 100)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #12293: [Enhancement](array-type) record offsets info to speed up array colum…
cambyzju commented on code in PR #12293:
URL: https://github.com/apache/doris/pull/12293#discussion_r965861465
##
be/src/olap/column_vector.cpp:
##
@@ -224,6 +224,16 @@ Status ArrayColumnVectorBatch::resize(size_t new_cap) {
return Status::OK();
}
+void ArrayColumnVectorBatch::put_item_ordinal(segment_v2::ordinal_t* ordinals,
size_t start_idx,
+ size_t size) {
+size_t first_offset = *(_offsets->scalar_cell_ptr(start_idx));
+segment_v2::ordinal_t first_ordinal = ordinals[0];
Review Comment:
need DCHECK(size > 0) or put this line inside while loop, do not worry about
performance, the compiler will cover it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #12474: [regression](array-type) add some case for array insert
github-actions[bot] commented on PR #12474: URL: https://github.com/apache/doris/pull/12474#issuecomment-1240619418 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #12474: [regression](array-type) add some case for array insert
github-actions[bot] commented on PR #12474: URL: https://github.com/apache/doris/pull/12474#issuecomment-1240619462 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #12463: [Enhancement](array-type) Add readable information in subquery for array type
github-actions[bot] commented on PR #12463: URL: https://github.com/apache/doris/pull/12463#issuecomment-1240620489 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #12463: [Enhancement](array-type) Add readable information in subquery for array type
github-actions[bot] commented on PR #12463: URL: https://github.com/apache/doris/pull/12463#issuecomment-1240620462 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon commented on a diff in pull request #12293: [Enhancement](array-type) record offsets info to speed up array colum…
eldenmoon commented on code in PR #12293:
URL: https://github.com/apache/doris/pull/12293#discussion_r965864261
##
be/src/olap/column_vector.cpp:
##
@@ -224,6 +224,16 @@ Status ArrayColumnVectorBatch::resize(size_t new_cap) {
return Status::OK();
}
+void ArrayColumnVectorBatch::put_item_ordinal(segment_v2::ordinal_t* ordinals,
size_t start_idx,
+ size_t size) {
+size_t first_offset = *(_offsets->scalar_cell_ptr(start_idx));
+segment_v2::ordinal_t first_ordinal = ordinals[0];
Review Comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yongjinhou commented on pull request #12321: [Function](ELT)Add elt function for doris
yongjinhou commented on PR #12321: URL: https://github.com/apache/doris/pull/12321#issuecomment-1240620683 > @yangzhg Hello，I would like to ask how to generate the newly added function signature by name mangling? nm doris_be | grep elt | grep FunctionContext Run the preceding command. doris_be is the binary generated by compiling BE elt is function name -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on pull request #12468: [feature-wip](nereids) implement uncheckedCast method in VarcharLiteral
Kikyou1997 commented on PR #12468: URL: https://github.com/apache/doris/pull/12468#issuecomment-1240621094 > please add regression test added unit test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon commented on a diff in pull request #12293: [Enhancement](array-type) record offsets info to speed up array colum…
eldenmoon commented on code in PR #12293: URL: https://github.com/apache/doris/pull/12293#discussion_r965865698 ## be/src/olap/rowset/segment_v2/column_reader.cpp: ## @@ -410,18 +410,33 @@ Status ArrayFileColumnIterator::next_batch(size_t* n, ColumnBlockView* dst, bool ColumnBlock* array_block = dst->column_block(); auto* array_batch = static_cast(array_block->vector_batch()); -// 1. read n offsets +// 1. read n+1 offsets +array_batch->offsets()->resize(*n + 1); ColumnBlock offset_block(array_batch->offsets(), nullptr); -ColumnBlockView offset_view(&offset_block, -dst->current_offset() + 1); // offset应该比collection的游标多1 +ColumnBlockView offset_view(&offset_block); Review Comment: but offset_view's first element may not be zero -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon commented on a diff in pull request #12293: [Enhancement](array-type) record offsets info to speed up array colum…
eldenmoon commented on code in PR #12293: URL: https://github.com/apache/doris/pull/12293#discussion_r965716438 ## be/src/common/config.h: ## @@ -849,6 +849,8 @@ CONF_Bool(enable_simdjson_reader, "false"); CONF_Int32(doris_remote_scanner_thread_pool_thread_num, "16"); // number of s3 scanner thread pool queue size CONF_Int32(doris_remote_scanner_thread_pool_queue_size, "10240"); +// write offsets instead of length to speed up seek performance +CONF_Bool(enable_write_array_offset, "false"); Review Comment: I had a conversation with @cambyzju and decided to only use `offset version`, so this PR is incompatible with previous. new Performace Test added to https://github.com/apache/doris/issues/12246 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangshuo128 commented on a diff in pull request #12481: (feature)(Nereids) Support function registry
wangshuo128 commented on code in PR #12481:
URL: https://github.com/apache/doris/pull/12481#discussion_r965867603
##
fe/fe-core/src/main/java/org/apache/doris/catalog/FunctionRegistry.java:
##
@@ -0,0 +1,176 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog;
+
+import org.apache.doris.nereids.annotation.Developing;
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.functions.AggregateFunction;
+import org.apache.doris.nereids.trees.expressions.functions.Avg;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.functions.Count;
+import org.apache.doris.nereids.trees.expressions.functions.FunctionBuilder;
+import org.apache.doris.nereids.trees.expressions.functions.Max;
+import org.apache.doris.nereids.trees.expressions.functions.Min;
+import org.apache.doris.nereids.trees.expressions.functions.ScalarFunction;
+import org.apache.doris.nereids.trees.expressions.functions.Substring;
+import org.apache.doris.nereids.trees.expressions.functions.Sum;
+import org.apache.doris.nereids.trees.expressions.functions.WeekOfYear;
+import org.apache.doris.nereids.trees.expressions.functions.Year;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableList;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+import javax.annotation.concurrent.ThreadSafe;
+
+/**
+ * New function registry for nereids.
+ *
+ * this class is developing for more functions.
+ */
+@Developing
+@ThreadSafe
+public class FunctionRegistry {
+public static final List SCALAR_FUNCTIONS = ImmutableList.of(
+scalar(Substring.class, "substr", "substring"),
+scalar(WeekOfYear.class),
+scalar(Year.class)
+);
+
+public static final ImmutableList AGGREGATE_FUNCTIONS =
ImmutableList.of(
+agg(Avg.class),
+agg(Count.class),
+agg(Max.class),
+agg(Min.class),
+agg(Sum.class)
+);
Review Comment:
We'd better have a dedicated class to hold these functions to let developers
add functions easily, such as `BuiltinFuncions.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] cambyzju commented on a diff in pull request #12293: [Enhancement](array-type) record offsets info to speed up array colum…
cambyzju commented on code in PR #12293:
URL: https://github.com/apache/doris/pull/12293#discussion_r965870153
##
be/src/olap/rowset/segment_v2/column_reader.cpp:
##
@@ -410,18 +410,33 @@ Status ArrayFileColumnIterator::next_batch(size_t* n,
ColumnBlockView* dst, bool
ColumnBlock* array_block = dst->column_block();
auto* array_batch =
static_cast(array_block->vector_batch());
-// 1. read n offsets
+// 1. read n+1 offsets
+array_batch->offsets()->resize(*n + 1);
ColumnBlock offset_block(array_batch->offsets(), nullptr);
-ColumnBlockView offset_view(&offset_block,
-dst->current_offset() + 1); //
offset应该比collection的游标多1
+ColumnBlockView offset_view(&offset_block);
bool offset_has_null = false;
-RETURN_IF_ERROR(_length_iterator->next_batch(n, &offset_view,
&offset_has_null));
+RETURN_IF_ERROR(_offset_iterator->next_batch(n, &offset_view,
&offset_has_null));
DCHECK(!offset_has_null);
if (*n == 0) {
return Status::OK();
}
-array_batch->get_offset_by_length(dst->current_offset(), *n);
+
+if (_offset_iterator->get_current_page()->has_remaining()) {
+// peek read one more to get the last array's length
+size_t i = 1;
+bool offsets_has_null = false;
+ordinal_t save = _offset_iterator->get_current_ordinal();
+RETURN_IF_ERROR(_offset_iterator->next_batch(&i, &offset_view,
&offsets_has_null));
Review Comment:
should we use another function to replace next_batch, to avoid twice call of
next_batch?
for example, we may has a function:
next_batch_for_offsets() {
// after normal read, always append more offset at the end;
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon commented on a diff in pull request #12293: [Enhancement](array-type) record offsets info to speed up array colum…
eldenmoon commented on code in PR #12293:
URL: https://github.com/apache/doris/pull/12293#discussion_r965875621
##
be/src/olap/rowset/segment_v2/column_reader.cpp:
##
@@ -410,18 +410,33 @@ Status ArrayFileColumnIterator::next_batch(size_t* n,
ColumnBlockView* dst, bool
ColumnBlock* array_block = dst->column_block();
auto* array_batch =
static_cast(array_block->vector_batch());
-// 1. read n offsets
+// 1. read n+1 offsets
+array_batch->offsets()->resize(*n + 1);
ColumnBlock offset_block(array_batch->offsets(), nullptr);
-ColumnBlockView offset_view(&offset_block,
-dst->current_offset() + 1); //
offset应该比collection的游标多1
+ColumnBlockView offset_view(&offset_block);
bool offset_has_null = false;
-RETURN_IF_ERROR(_length_iterator->next_batch(n, &offset_view,
&offset_has_null));
+RETURN_IF_ERROR(_offset_iterator->next_batch(n, &offset_view,
&offset_has_null));
DCHECK(!offset_has_null);
if (*n == 0) {
return Status::OK();
}
-array_batch->get_offset_by_length(dst->current_offset(), *n);
+
+if (_offset_iterator->get_current_page()->has_remaining()) {
+// peek read one more to get the last array's length
+size_t i = 1;
+bool offsets_has_null = false;
+ordinal_t save = _offset_iterator->get_current_ordinal();
+RETURN_IF_ERROR(_offset_iterator->next_batch(&i, &offset_view,
&offsets_has_null));
Review Comment:
agree, may be `peek_batch_for_offsets`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] TaoZex commented on pull request #12321: [Function](ELT)Add elt function for doris
TaoZex commented on PR #12321: URL: https://github.com/apache/doris/pull/12321#issuecomment-1240644363 > > @yangzhg Hello，I would like to ask how to generate the newly added function signature by name mangling? > > nm doris_be | grep elt | grep FunctionContext > > Run the preceding command. > > doris_be is the binary generated by compiling BE > > elt is function name What are the following steps in the development process? 1. Build a development environment 3. Write function source code and put it in BE 4. Compile BE and generate function signature through name mangling 5. Test the code function (but I don't know how to test it yet) 6. Submit pr -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on a diff in pull request #12483: [enhancement](Nereids) add all necessary PhysicalDistribute in CostAndEnforcerJob
jackwener commented on code in PR #12483:
URL: https://github.com/apache/doris/pull/12483#discussion_r965910325
##
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/cascades/CostAndEnforcerJob.java:
##
@@ -184,33 +186,47 @@ public void execute() {
curTotalCost -= curNodeCost;
curNodeCost = CostCalculator.calculateCost(groupExpression);
curTotalCost += curNodeCost;
-// record map { outputProperty -> outputProperty }, { ANY ->
outputProperty },
-recordPropertyAndCost(groupExpression, outputProperty,
outputProperty, requestChildrenProperty);
-recordPropertyAndCost(groupExpression, outputProperty,
PhysicalProperties.ANY, requestChildrenProperty);
-enforce(outputProperty, requestChildrenProperty);
+// colocate join and bucket shuffle join cannot add enforce to
satisfy required property.
+if (enforce(outputProperty, requestChildrenProperty)) {
-if (curTotalCost < context.getCostUpperBound()) {
-context.setCostUpperBound(curTotalCost);
+// record map { outputProperty -> outputProperty }, { ANY
-> outputProperty },
+recordPropertyAndCost(groupExpression, outputProperty,
outputProperty, requestChildrenProperty);
+recordPropertyAndCost(groupExpression, outputProperty,
PhysicalProperties.ANY,
+requestChildrenProperty);
Review Comment:
recordPropertyAndCost may be success instead of putting inside if.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow opened a new pull request, #12486: [fix](Nereids) column prune generate empty project list on join's child
morrySnow opened a new pull request, #12486: URL: https://github.com/apache/doris/pull/12486 # Proposed changes Issue Number: close #xxx ## Problem summary When we do cross join and project on it only has one side of its child, we will generate a empty project on its child and lead be core dump. This PR fix this problem. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on a diff in pull request #12483: [enhancement](Nereids) add all necessary PhysicalDistribute in CostAndEnforcerJob
morrySnow commented on code in PR #12483:
URL: https://github.com/apache/doris/pull/12483#discussion_r965934727
##
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/cascades/CostAndEnforcerJob.java:
##
@@ -184,33 +186,47 @@ public void execute() {
curTotalCost -= curNodeCost;
curNodeCost = CostCalculator.calculateCost(groupExpression);
curTotalCost += curNodeCost;
-// record map { outputProperty -> outputProperty }, { ANY ->
outputProperty },
-recordPropertyAndCost(groupExpression, outputProperty,
outputProperty, requestChildrenProperty);
-recordPropertyAndCost(groupExpression, outputProperty,
PhysicalProperties.ANY, requestChildrenProperty);
-enforce(outputProperty, requestChildrenProperty);
+// colocate join and bucket shuffle join cannot add enforce to
satisfy required property.
+if (enforce(outputProperty, requestChildrenProperty)) {
-if (curTotalCost < context.getCostUpperBound()) {
-context.setCostUpperBound(curTotalCost);
+// record map { outputProperty -> outputProperty }, { ANY
-> outputProperty },
+recordPropertyAndCost(groupExpression, outputProperty,
outputProperty, requestChildrenProperty);
+recordPropertyAndCost(groupExpression, outputProperty,
PhysicalProperties.ANY,
+requestChildrenProperty);
Review Comment:
enforce return false means this branch is not work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yongjinhou commented on pull request #12321: [Function](ELT)Add elt function for doris
yongjinhou commented on PR #12321: URL: https://github.com/apache/doris/pull/12321#issuecomment-1240682599 > > > @yangzhg Hello，I would like to ask how to generate the newly added function signature by name mangling? > > > nm doris_be | grep elt | grep FunctionContext > > > Run the preceding command. > > > doris_be is the binary generated by compiling BE > > > elt is function name > > What are the following steps in the development process? > > 1. Build a development environment > 2. Write function source code and put it in BE > 3. Compile BE and generate function signature through name mangling > 4. Test the code function (but I don't know how to test it yet) > 5. Submit pr you can click "https://www.bilibili.com/video/BV1V44y1Y7Ro/?spm_id_from=333.788&vd_source=897445b3e6d43614801a4aa3f1b3b57c"; to learn more and see "Files changed" in this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on pull request #12486: [fix](Nereids) column prune generate empty project list on join's child
Kikyou1997 commented on PR #12486: URL: https://github.com/apache/doris/pull/12486#issuecomment-1240682957 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] TaoZex commented on pull request #12321: [Function](ELT)Add elt function for doris
TaoZex commented on PR #12321: URL: https://github.com/apache/doris/pull/12321#issuecomment-1240685903 > > > > @yangzhg Hello，I would like to ask how to generate the newly added function signature by name mangling? > > > > nm doris_be | grep elt | grep FunctionContext > > > > Run the preceding command. > > > > doris_be is the binary generated by compiling BE > > > > elt is function name > > > > > > What are the following steps in the development process? > > > > 1. Build a development environment > > 2. Write function source code and put it in BE > > 3. Compile BE and generate function signature through name mangling > > 4. Test the code function (but I don't know how to test it yet) > > 5. Submit pr > > you can click "https://www.bilibili.com/video/BV1V44y1Y7Ro/?spm_id_from=333.788&vd_source=897445b3e6d43614801a4aa3f1b3b57c"; to learn more and see "Files changed" in this PR. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon commented on a diff in pull request #12293: [Enhancement](array-type) record offsets info to speed up array colum…
eldenmoon commented on code in PR #12293:
URL: https://github.com/apache/doris/pull/12293#discussion_r965946299
##
be/src/olap/rowset/segment_v2/column_reader.cpp:
##
@@ -410,18 +410,33 @@ Status ArrayFileColumnIterator::next_batch(size_t* n,
ColumnBlockView* dst, bool
ColumnBlock* array_block = dst->column_block();
auto* array_batch =
static_cast(array_block->vector_batch());
-// 1. read n offsets
+// 1. read n+1 offsets
+array_batch->offsets()->resize(*n + 1);
ColumnBlock offset_block(array_batch->offsets(), nullptr);
-ColumnBlockView offset_view(&offset_block,
-dst->current_offset() + 1); //
offset应该比collection的游标多1
+ColumnBlockView offset_view(&offset_block);
bool offset_has_null = false;
-RETURN_IF_ERROR(_length_iterator->next_batch(n, &offset_view,
&offset_has_null));
+RETURN_IF_ERROR(_offset_iterator->next_batch(n, &offset_view,
&offset_has_null));
DCHECK(!offset_has_null);
if (*n == 0) {
return Status::OK();
}
-array_batch->get_offset_by_length(dst->current_offset(), *n);
+
+if (_offset_iterator->get_current_page()->has_remaining()) {
+// peek read one more to get the last array's length
+size_t i = 1;
+bool offsets_has_null = false;
+ordinal_t save = _offset_iterator->get_current_ordinal();
+RETURN_IF_ERROR(_offset_iterator->next_batch(&i, &offset_view,
&offsets_has_null));
Review Comment:
sorry, I think conbining the two next_batch is a little bit confusing, since
the second one is a `peek` operator, while the first is `next` operation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon commented on a diff in pull request #12293: [Enhancement](array-type) record offsets info to speed up array colum…
eldenmoon commented on code in PR #12293:
URL: https://github.com/apache/doris/pull/12293#discussion_r965875621
##
be/src/olap/rowset/segment_v2/column_reader.cpp:
##
@@ -410,18 +410,33 @@ Status ArrayFileColumnIterator::next_batch(size_t* n,
ColumnBlockView* dst, bool
ColumnBlock* array_block = dst->column_block();
auto* array_batch =
static_cast(array_block->vector_batch());
-// 1. read n offsets
+// 1. read n+1 offsets
+array_batch->offsets()->resize(*n + 1);
ColumnBlock offset_block(array_batch->offsets(), nullptr);
-ColumnBlockView offset_view(&offset_block,
-dst->current_offset() + 1); //
offset应该比collection的游标多1
+ColumnBlockView offset_view(&offset_block);
bool offset_has_null = false;
-RETURN_IF_ERROR(_length_iterator->next_batch(n, &offset_view,
&offset_has_null));
+RETURN_IF_ERROR(_offset_iterator->next_batch(n, &offset_view,
&offset_has_null));
DCHECK(!offset_has_null);
if (*n == 0) {
return Status::OK();
}
-array_batch->get_offset_by_length(dst->current_offset(), *n);
+
+if (_offset_iterator->get_current_page()->has_remaining()) {
+// peek read one more to get the last array's length
+size_t i = 1;
+bool offsets_has_null = false;
+ordinal_t save = _offset_iterator->get_current_ordinal();
+RETURN_IF_ERROR(_offset_iterator->next_batch(&i, &offset_view,
&offsets_has_null));
Review Comment:
agree, may be `peek_batch_for_offsets`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jacktengg opened a new pull request, #12487: [bugfix](odbc) escape identifiers for sqlserver and postgresql
jacktengg opened a new pull request, #12487:
URL: https://github.com/apache/doris/pull/12487
# Proposed changes
Issue Number: close #xxx
## Problem summary
Delimited identifier format for sqlserver and postgresql is different from
MySQL.
Sqlserver use brackets ([ ]) and postgresql use double quotes("").
## Checklist(Required)
1. Does it affect the original behavior:
- [ ] Yes
- [ ] No
- [ ] I don't know
2. Has unit tests been added:
- [ ] Yes
- [ ] No
- [ ] No Need
3. Has document been added or modified:
- [ ] Yes
- [ ] No
- [ ] No Need
4. Does it need to update dependencies:
- [ ] Yes
- [ ] No
5. Are there any changes that cannot be rolled back:
- [ ] Yes (If Yes, please explain WHY)
- [ ] No
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] 924060929 commented on a diff in pull request #12481: (feature)(Nereids) Support function registry
924060929 commented on code in PR #12481:
URL: https://github.com/apache/doris/pull/12481#discussion_r965963869
##
fe/fe-core/src/main/java/org/apache/doris/catalog/FunctionRegistry.java:
##
@@ -0,0 +1,176 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog;
+
+import org.apache.doris.nereids.annotation.Developing;
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.functions.AggregateFunction;
+import org.apache.doris.nereids.trees.expressions.functions.Avg;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.functions.Count;
+import org.apache.doris.nereids.trees.expressions.functions.FunctionBuilder;
+import org.apache.doris.nereids.trees.expressions.functions.Max;
+import org.apache.doris.nereids.trees.expressions.functions.Min;
+import org.apache.doris.nereids.trees.expressions.functions.ScalarFunction;
+import org.apache.doris.nereids.trees.expressions.functions.Substring;
+import org.apache.doris.nereids.trees.expressions.functions.Sum;
+import org.apache.doris.nereids.trees.expressions.functions.WeekOfYear;
+import org.apache.doris.nereids.trees.expressions.functions.Year;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableList;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+import javax.annotation.concurrent.ThreadSafe;
+
+/**
+ * New function registry for nereids.
+ *
+ * this class is developing for more functions.
+ */
+@Developing
+@ThreadSafe
+public class FunctionRegistry {
+public static final List SCALAR_FUNCTIONS = ImmutableList.of(
+scalar(Substring.class, "substr", "substring"),
+scalar(WeekOfYear.class),
+scalar(Year.class)
+);
+
+public static final ImmutableList AGGREGATE_FUNCTIONS =
ImmutableList.of(
+agg(Avg.class),
+agg(Count.class),
+agg(Max.class),
+agg(Min.class),
+agg(Sum.class)
+);
Review Comment:
The function in the lists is always builtin function, add a builtin function
means add the function in this lists.
So what specify is the `BuiltinFunctions`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] 924060929 commented on a diff in pull request #12481: (feature)(Nereids) Support function registry
924060929 commented on code in PR #12481:
URL: https://github.com/apache/doris/pull/12481#discussion_r965963869
##
fe/fe-core/src/main/java/org/apache/doris/catalog/FunctionRegistry.java:
##
@@ -0,0 +1,176 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog;
+
+import org.apache.doris.nereids.annotation.Developing;
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.functions.AggregateFunction;
+import org.apache.doris.nereids.trees.expressions.functions.Avg;
+import org.apache.doris.nereids.trees.expressions.functions.BoundFunction;
+import org.apache.doris.nereids.trees.expressions.functions.Count;
+import org.apache.doris.nereids.trees.expressions.functions.FunctionBuilder;
+import org.apache.doris.nereids.trees.expressions.functions.Max;
+import org.apache.doris.nereids.trees.expressions.functions.Min;
+import org.apache.doris.nereids.trees.expressions.functions.ScalarFunction;
+import org.apache.doris.nereids.trees.expressions.functions.Substring;
+import org.apache.doris.nereids.trees.expressions.functions.Sum;
+import org.apache.doris.nereids.trees.expressions.functions.WeekOfYear;
+import org.apache.doris.nereids.trees.expressions.functions.Year;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableList;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+import javax.annotation.concurrent.ThreadSafe;
+
+/**
+ * New function registry for nereids.
+ *
+ * this class is developing for more functions.
+ */
+@Developing
+@ThreadSafe
+public class FunctionRegistry {
+public static final List SCALAR_FUNCTIONS = ImmutableList.of(
+scalar(Substring.class, "substr", "substring"),
+scalar(WeekOfYear.class),
+scalar(Year.class)
+);
+
+public static final ImmutableList AGGREGATE_FUNCTIONS =
ImmutableList.of(
+agg(Avg.class),
+agg(Count.class),
+agg(Max.class),
+agg(Min.class),
+agg(Sum.class)
+);
Review Comment:
The function in the lists is always builtin function, add a builtin function
means add the function in this lists.
So What is special about `BuiltinFunctions`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] AshinGau opened a new pull request, #12488: [feature-wip](parquet-reader) add gzip compression codec
AshinGau opened a new pull request, #12488: URL: https://github.com/apache/doris/pull/12488 # Proposed changes ## Problem summary Query failed when read parquet data compressed by GZIP: ``` mysql> select * from customer limit 1; ERROR 1105 (HY000): errCode = 2, detailMessage = unknown compression type(GZIP) ``` ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [x] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [x] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener opened a new pull request, #12489: [fix](Nereids): fix having in Nereids regress test.
jackwener opened a new pull request, #12489: URL: https://github.com/apache/doris/pull/12489 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
