This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch branch-1.2-unstable
in repository https://gitbox.apache.org/repos/asf/doris.git
from 753c2ccfd1 [fix](test) drop table before create it (#13791)
add b15e0a9fb5 [Bug](function) fix bug of if function of nullable column
process (#13779)
add 6159e1cc3a [enhancement](tpch-tools) git ignore tpch tool gen file
#13789
add 711dad28fb [Chore](unused) remove QSorter #13769
add f5761c658f [Fix]Fix the extension mysql_to_doris bug (#13723)
add 2b9e1878a2 [fix](hashjoin) return error if in progress of upgrade
(#13753)
add 61b7c2c96c [fix](join) fix incorrect result when using anti join with
other join predicates (#13743)
add 53e5f3939e [fix](plan)result exprs should be substituted in the same
way as agg exprs (#13744)
add 4f2ea0776c [enhancement](compaction) opt compaction task producer and
quick compaction (#13495)
add 2fb218173e [improvement](scan) change the max thread num and num of
free blocks in new scan (#13793)
add ceb7b60a64 [fix](Nereids) update immutable LogicalAggregate attribute
by mistake (#13740)
add f49a0daf54 [fix](regression) Fix concurrent regression failure (#13798)
add ba177a15cb [feature-wip](recover) new recover ddl and support show
catalog recycle bin (#13067)
add 18be77af64 [fix](nereids) query cannot execution when both nereids
enable and fallback to legacy planner are set to false (#13787)
add 7ae60a0ad2 [feature](function)add url functions: domain and protocol
(#13662)
add 57a9b0fa65 [Enhancement](chore) remove unused diagnostic (#12337)
add 36a47dfe16 [enhancement](Nereids): use ImmutableList explicitly in
Plan (#13817)
add cc0fa5fef6 [fix](array-type) fix the be core dump when import
array<largeint> (#13821)
add d2c5c1af3b [feature](regression) add custom config file for
Regression: regression-conf-custom.groovy (#13783)
add b27714542d [fix](planner) infer predicate could generate predicates in
another scope (#13691)
add 7f2166b1fd [fix](thrift) fix that thrift struct sequence number is not
consistent in 1.1-lts and master (#13829)
add 164ca1e1a8 [Bug](function) change log fatal to log warning to avoid
code dump on nullable double column cast to decimal column (#13819)
add 34e68a41dd [enhancement](explain) add cardinality to explain string
and explain graph (#13720)
add 42b2725f03 [Bug](delete) Fix wrong delete operation (#13840)
add e63608b556 [Bug](test) fix some test case result is ramdom (#13837)
add 7db916fc85 [enhancement](metric)Add metric for exec_state prepare
function (#13646)
add 942611c185 Revert "[enhancement](compaction) opt compaction task
producer and quick compaction (#13495)" (#13833)
add 83e55cade8 [feature](Nereids): add rule for matching plan into
HyperGraph. (#13805)
add c14277e587 [fix](analytic) fix coredump cause by empty analytic
parameter types (#13808)
add f30b974d54 [Bugfix](upgrade) Fix 1.1 upgrade 1.2 coredump when schema
change (#13822)
add 1eef986e75 [feature](nereids) add rule for semi/anti join exploration,
when there is project between them (#13756)
add 61c817f4cc [feature](syntax) support SELECT * EXCEPT (#13844)
add 01f9f8ad43 [enhancement](Nereids) add merge project rule to column
prune rule set (#13835)
add f0c9867af3 [fix](nereids) map literal to double in
FilterSelectivityCalculator (#13776)
add 7f34698eef [enhancement](Nereids) use join estimation v2 only when
stats derive v2 is enable (#13845)
add 287a739510 [javaudf](string) Fix string format in java udf (#13854)
add 8b3afd431e [improvement](memory) simplify memory config related to
tcmalloc (#13781)
add 3924ecead5 [minor](load) Improve error message for string type in
loading process (#13718)
add 7fedfdcf6a [fix](spark load)The where condition does not take effect
when spark load loads the file (#13803)
add de1dc62843 [enhancement](olap scanner) Scanner row bytes buffer is too
small bug (#13874)
add 0eeb4d2881 [minor](log) remove some e.printStackTrace() (#13870)
add 947e67fa76 [enhancement](test) retry start be or fe when port has been
bind. (#13860)
add 3fc1b27c40 [docs](tablet-docs) fix the tablet-repair-and-balance.md
doucument. (#13853)
add bd6070d9b3 [doc](spark-doris-connetor)Add spark Doris connector to
support streamload documentation #13834
add 277025b046 [fix](join)ColumnNullable need handle const column with
nullable const value (#13866)
add be124523f4 [enhancement](profile) add profile to show column
predicates (#13862)
add 667cfe5598 [community](collaborators) add more collaborators (#13880)
add d5becdb4a1 [fix](dynamic-partition) fix wrong check of replication num
(#13755)
add e6080a6e4c [regression](join) add right anti join with other predicate
regression case (#13815)
add ee8dffbfb7 [meta](recover) change dropInfo and RecoverInfo to GSON
(#13830)
add ba918b40e2 [chore](macOS) Fix compilation errors caused by the
deprecated function (#13890)
No new revisions were added by this update.
Summary of changes:
.asf.yaml | 2 +
.gitignore | 12 +-
be/src/common/config.h | 22 +-
be/src/common/daemon.cpp | 16 +-
be/src/exec/exec_node.cpp | 7 +
be/src/exec/olap_scanner.cpp | 14 +-
be/src/exprs/agg_fn_evaluator.cpp | 15 -
be/src/olap/bloom_filter_predicate.h | 6 +
be/src/olap/column_predicate.h | 55 +++
be/src/olap/comparison_predicate.h | 46 +++
be/src/olap/in_list_predicate.h | 28 ++
be/src/olap/iterators.h | 2 +
be/src/olap/like_column_predicate.h | 5 +
be/src/olap/null_predicate.h | 13 +
be/src/olap/reader.h | 2 +
be/src/olap/rowset/beta_rowset_reader.h | 7 +
be/src/olap/rowset/rowset_reader.h | 2 +
be/src/olap/rowset/segment_v2/column_reader.cpp | 6 +-
be/src/olap/rowset/segment_v2/segment_iterator.h | 31 ++
be/src/olap/schema_change.cpp | 6 +
be/src/runtime/CMakeLists.txt | 1 -
be/src/runtime/fragment_mgr.cpp | 10 +-
be/src/runtime/qsorter.cpp | 137 --------
be/src/runtime/qsorter.h | 75 -----
be/src/service/doris_main.cpp | 22 +-
be/src/util/array_parser.h | 2 +-
be/src/util/types.h | 3 -
be/src/vec/CMakeLists.txt | 1 +
.../aggregate_function_java_udaf.h | 4 +
be/src/vec/columns/column_nullable.cpp | 7 +
be/src/vec/common/allocator.h | 13 -
be/src/vec/common/uint128.h | 20 --
be/src/vec/core/field.h | 8 -
be/src/vec/data_types/data_type_decimal.cpp | 12 +-
be/src/vec/data_types/data_type_decimal.h | 10 +-
be/src/vec/exec/format/json/new_json_reader.cpp | 10 +-
be/src/vec/exec/scan/new_olap_scan_node.cpp | 6 +-
be/src/vec/exec/scan/new_olap_scanner.cpp | 13 +-
be/src/vec/exec/scan/new_olap_scanner.h | 6 +-
be/src/vec/exec/scan/scanner_context.cpp | 28 +-
be/src/vec/exprs/vectorized_agg_fn.cpp | 24 +-
be/src/vec/functions/function_java_udf.cpp | 20 +-
be/src/vec/functions/if.cpp | 12 +-
be/src/vec/functions/simple_function_factory.h | 3 +
be/src/vec/functions/url/domain.h | 148 ++++++++
be/src/vec/functions/url/function_url.cpp | 53 +++
be/src/vec/functions/url/functions_url.h | 165 +++++++++
be/src/vec/functions/url/protocol.h | 63 ++++
be/src/vec/olap/block_reader.h | 4 +
be/src/vec/olap/vcollect_iterator.h | 23 ++
be/src/vec/olap/vgeneric_iterators.cpp | 14 +
be/src/vec/sink/vtablet_sink.cpp | 6 +-
be/test/CMakeLists.txt | 2 +-
be/test/runtime/qsorter_test.cpp | 279 ----------------
be/test/vec/function/function_url_test.cpp | 92 +++++
docs/en/docs/admin-manual/config/be-config.md | 26 +-
.../maint-monitor/tablet-repair-and-balance.md | 2 +-
.../docs/advanced/partition/dynamic-partition.md | 85 ++++-
docs/en/docs/data-table/basic-usage.md | 18 +
docs/en/docs/ecosystem/spark-doris-connector.md | 2 +
.../Create/CREATE-TABLE.md | 2 +-
docs/zh-CN/docs/admin-manual/config/be-config.md | 26 +-
.../maint-monitor/tablet-repair-and-balance.md | 4 +-
.../docs/advanced/partition/dynamic-partition.md | 81 +++++
docs/zh-CN/docs/data-table/basic-usage.md | 18 +
docs/zh-CN/docs/ecosystem/spark-doris-connector.md | 1 +
.../Create/CREATE-TABLE.md | 2 +-
extension/mysql_to_doris/bin/e_mysql_to_doris.sh | 4 +-
.../org/apache/doris/common/FeMetaVersion.java | 4 +-
fe/fe-core/src/main/cup/sql_parser.cup | 52 ++-
.../doris/alter/MaterializedViewHandler.java | 2 +-
.../java/org/apache/doris/analysis/Analyzer.java | 172 ++++++----
.../org/apache/doris/analysis/FunctionParams.java | 4 +-
.../org/apache/doris/analysis/RecoverDbStmt.java | 34 +-
.../doris/analysis/RecoverPartitionStmt.java | 27 +-
.../apache/doris/analysis/RecoverTableStmt.java | 24 +-
.../java/org/apache/doris/analysis/SelectList.java | 12 +
.../java/org/apache/doris/analysis/SelectStmt.java | 95 ++++--
.../doris/analysis/ShowCatalogRecycleBinStmt.java | 154 +++++++++
.../apache/doris/catalog/CatalogRecycleBin.java | 372 ++++++++++++++-------
.../main/java/org/apache/doris/catalog/Env.java | 28 +-
.../java/org/apache/doris/catalog/OlapTable.java | 8 +-
.../doris/clone/DynamicPartitionScheduler.java | 2 +-
.../java/org/apache/doris/common/GenericPool.java | 2 +-
.../doris/common/util/DynamicPartitionUtil.java | 19 +-
.../org/apache/doris/common/util/SmallFileMgr.java | 3 +-
.../apache/doris/datasource/InternalCatalog.java | 152 ++++++---
.../org/apache/doris/journal/JournalEntity.java | 9 +-
.../org/apache/doris/nereids/NereidsPlanner.java | 4 +-
.../glue/translator/PhysicalPlanTranslator.java | 21 +-
.../apache/doris/nereids/memo/GroupExpression.java | 12 +-
.../java/org/apache/doris/nereids/memo/Memo.java | 13 +-
.../org/apache/doris/nereids/rules/RuleSet.java | 2 +
.../org/apache/doris/nereids/rules/RuleType.java | 1 +
.../rules/exploration/join/JoinReorderContext.java | 11 +
.../join/SemiJoinSemiJoinTransposeProject.java | 99 ++++++
.../HyperGraphJoinReorder.java} | 31 +-
.../HyperGraphJoinReorderGroupPlan.java} | 30 +-
.../join => joinreorder}/hypergraph/Edge.java | 2 +-
.../hypergraph/HyperGraph.java | 30 +-
.../join => joinreorder}/hypergraph/Node.java | 44 ++-
.../join => joinreorder}/hypergraph/Receiver.java | 6 +-
.../rules/rewrite/AggregateDisassemble.java | 3 +-
.../rewrite/logical/ApplyPullFilterOnAgg.java | 3 +-
.../logical/ApplyPullFilterOnProjectUnderAgg.java | 13 +-
.../rules/rewrite/logical/ColumnPruning.java | 3 +-
.../rules/rewrite/logical/NormalizeAggregate.java | 12 +-
.../nereids/stats/FilterSelectivityCalculator.java | 5 +-
.../apache/doris/nereids/stats/JoinEstimation.java | 7 +-
.../trees/expressions/literal/BooleanLiteral.java | 9 +
.../trees/expressions/literal/DecimalLiteral.java | 5 +
.../trees/expressions/literal/LargeIntLiteral.java | 5 +
.../nereids/trees/expressions/literal/Literal.java | 6 +-
.../trees/expressions/literal/NullLiteral.java | 5 +
.../trees/plans/logical/LogicalAggregate.java | 20 +-
.../nereids/trees/plans/logical/LogicalApply.java | 2 +-
.../trees/plans/logical/LogicalEmptyRelation.java | 2 +-
.../nereids/trees/plans/logical/LogicalJoin.java | 8 +-
.../trees/plans/logical/LogicalOlapScan.java | 15 +-
.../trees/plans/logical/LogicalOneRowRelation.java | 2 +-
.../trees/plans/logical/LogicalProject.java | 2 +-
.../trees/plans/logical/LogicalRelation.java | 4 +-
.../trees/plans/logical/LogicalSelectHint.java | 2 +-
.../nereids/trees/plans/logical/LogicalSort.java | 4 +-
.../nereids/trees/plans/logical/LogicalTopN.java | 4 +-
.../trees/plans/physical/AbstractPhysicalJoin.java | 13 +-
.../trees/plans/physical/AbstractPhysicalSort.java | 2 +-
.../trees/plans/physical/PhysicalAggregate.java | 18 +-
.../trees/plans/physical/PhysicalBinary.java | 6 -
.../plans/physical/PhysicalEmptyRelation.java | 2 +-
.../trees/plans/physical/PhysicalHashJoin.java | 1 +
.../nereids/trees/plans/physical/PhysicalLeaf.java | 5 -
.../trees/plans/physical/PhysicalOlapScan.java | 16 +-
.../plans/physical/PhysicalOneRowRelation.java | 2 +-
.../trees/plans/physical/PhysicalProject.java | 7 +-
.../trees/plans/physical/PhysicalRelation.java | 6 +-
.../trees/plans/physical/RuntimeFilter.java | 4 +-
.../java/org/apache/doris/nereids/util/Utils.java | 2 +-
.../java/org/apache/doris/persist/DropDbInfo.java | 14 +-
.../java/org/apache/doris/persist/DropInfo.java | 45 ++-
.../apache/doris/persist/DropPartitionInfo.java | 12 +-
.../java/org/apache/doris/persist/EditLog.java | 7 +-
.../java/org/apache/doris/persist/RecoverInfo.java | 51 ++-
.../org/apache/doris/planner/AggregationNode.java | 14 +-
.../org/apache/doris/planner/AnalyticEvalNode.java | 6 +-
.../apache/doris/planner/AssertNumRowsNode.java | 2 +-
.../org/apache/doris/planner/CrossJoinNode.java | 12 +-
.../org/apache/doris/planner/EmptySetNode.java | 1 +
.../org/apache/doris/planner/HashJoinNode.java | 1 +
.../org/apache/doris/planner/OlapScanNode.java | 1 +
.../java/org/apache/doris/planner/PlanNode.java | 4 +
.../java/org/apache/doris/planner/SelectNode.java | 2 +-
.../org/apache/doris/planner/SetOperationNode.java | 4 +-
.../apache/doris/planner/SingleNodePlanner.java | 7 +-
.../java/org/apache/doris/planner/SortNode.java | 6 +-
.../apache/doris/planner/TableFunctionNode.java | 13 +-
.../java/org/apache/doris/qe/ConnectProcessor.java | 9 +-
.../java/org/apache/doris/qe/ShowExecutor.java | 15 +
.../java/org/apache/doris/qe/StmtExecutor.java | 1 +
.../java/org/apache/doris/rewrite/FEFunctions.java | 1 -
fe/fe-core/src/main/jflex/sql_scanner.flex | 2 +
.../org/apache/doris/analysis/SelectStmtTest.java | 9 +
.../java/org/apache/doris/catalog/DropDbTest.java | 2 +-
.../apache/doris/catalog/DropPartitionTest.java | 2 +-
.../org/apache/doris/catalog/DropTableTest.java | 2 +-
.../doris/catalog/DynamicPartitionTableTest.java | 34 +-
.../apache/doris/catalog/ModifyBackendTest.java | 2 +-
.../java/org/apache/doris/catalog/RecoverTest.java | 122 +++++--
.../org/apache/doris/nereids/memo/MemoTest.java | 6 +-
.../join/SemiJoinSemiJoinTransposeProjectTest.java | 77 +++++
.../HyperGraphJoinReorderGroupPlanTest.java} | 36 +-
.../HyperGraphJoinReorderTest.java} | 40 +--
.../hypergraph/HyperGraphTest.java | 25 +-
.../rewrite/logical/AggregateDisassembleTest.java | 282 +++++++---------
.../doris/nereids/stats/StatsCalculatorTest.java | 12 +-
.../org/apache/doris/persist/DropDbInfoTest.java | 8 +-
.../org/apache/doris/persist/DropInfoTest.java | 10 +-
.../doris/persist/DropPartitionInfoTest.java | 14 +-
.../java/org/apache/doris/utframe/DorisAssert.java | 4 +-
.../org/apache/doris/utframe/UtFrameUtils.java | 27 +-
.../java/org/apache/doris/udf/UdafExecutor.java | 13 +-
.../java/org/apache/doris/udf/UdfExecutor.java | 21 +-
.../main/java/org/apache/doris/udf/UdfUtils.java | 1 -
.../java/org/apache/doris/udf/UdfExecutorTest.java | 15 +-
.../org/apache/doris/load/loadv2/dpp/SparkDpp.java | 3 +
gensrc/script/doris_builtins_functions.py | 11 +
gensrc/thrift/Exprs.thrift | 2 +-
gensrc/thrift/PaloInternalService.thrift | 10 +-
...n_with_cross_join.out => test_implict_cast.out} | 2 +-
.../test_outer_join_with_cross_join.out | 3 +
regression-test/data/ddl_p0/test_recover.out | 83 +++++
.../data/delete_p0/test_zone_map_delete.out | 59 ++++
.../data/javaudf_p0/test_javaudf_string.out | 37 ++
regression-test/data/query/join/test_join.out | 23 ++
.../conditional_functions/test_nullif.out | 18 +
.../window_functions/test_window_function.out | 30 +-
.../org/apache/doris/regression/Config.groovy | 11 +-
.../main/java/org/apache/doris/udf/StringTest.java | 11 +-
.../suites/correctness/test_infer_predicate.groovy | 114 +++++++
.../suites/correctness/test_select_except.groovy | 69 ++++
.../suites/correctness_p0/test_implict_cast.groovy | 59 ++++
.../test_outer_join_with_cross_join.groovy | 15 +
...t_outer_join_with_window_function_datev2.groovy | 34 +-
regression-test/suites/ddl_p0/test_recover.groovy | 268 +++++++++++++++
.../suites/delete_p0/test_zone_map_delete.groovy | 57 ++++
.../suites/javaudf_p0/test_javaudf_string.groovy | 71 ++++
regression-test/suites/query/join/test_join.groovy | 3 +
.../conditional_functions/test_nullif.groovy | 1 +
.../window_functions/test_window_function.groovy | 6 +-
209 files changed, 3973 insertions(+), 1591 deletions(-)
delete mode 100644 be/src/runtime/qsorter.cpp
delete mode 100644 be/src/runtime/qsorter.h
create mode 100644 be/src/vec/functions/url/domain.h
create mode 100644 be/src/vec/functions/url/function_url.cpp
create mode 100644 be/src/vec/functions/url/functions_url.h
create mode 100644 be/src/vec/functions/url/protocol.h
delete mode 100644 be/test/runtime/qsorter_test.cpp
create mode 100644 be/test/vec/function/function_url_test.cpp
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/analysis/ShowCatalogRecycleBinStmt.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/SemiJoinSemiJoinTransposeProject.java
copy
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/{exploration/join/JoinReorderRule.java
=> joinreorder/HyperGraphJoinReorder.java} (50%)
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/{exploration/join/JoinReorderRule.java
=> joinreorder/HyperGraphJoinReorderGroupPlan.java} (52%)
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/{exploration/join =>
joinreorder}/hypergraph/Edge.java (97%)
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/{exploration/join =>
joinreorder}/hypergraph/HyperGraph.java (84%)
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/{exploration/join =>
joinreorder}/hypergraph/Node.java (64%)
rename
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/{exploration/join =>
joinreorder}/hypergraph/Receiver.java (92%)
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/exploration/join/SemiJoinSemiJoinTransposeProjectTest.java
copy
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/{exploration/join/hypergraph/HyperGraphTest.java
=> joinreorder/HyperGraphJoinReorderGroupPlanTest.java} (60%)
copy
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/{exploration/join/hypergraph/HyperGraphTest.java
=> joinreorder/HyperGraphJoinReorderTest.java} (56%)
rename
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/{exploration/join =>
joinreorder}/hypergraph/HyperGraphTest.java (72%)
copy regression-test/data/correctness_p0/{test_outer_join_with_cross_join.out
=> test_implict_cast.out} (96%)
create mode 100644 regression-test/data/ddl_p0/test_recover.out
create mode 100644 regression-test/data/delete_p0/test_zone_map_delete.out
create mode 100644 regression-test/data/javaudf_p0/test_javaudf_string.out
rename
fe/fe-core/src/test/java/org/apache/doris/nereids/rules/exploration/join/JoinReorderTest.java
=>
regression-test/java-udf-src/src/main/java/org/apache/doris/udf/StringTest.java
(70%)
create mode 100644
regression-test/suites/correctness/test_infer_predicate.groovy
create mode 100644 regression-test/suites/correctness/test_select_except.groovy
create mode 100644
regression-test/suites/correctness_p0/test_implict_cast.groovy
create mode 100644 regression-test/suites/ddl_p0/test_recover.groovy
create mode 100644 regression-test/suites/delete_p0/test_zone_map_delete.groovy
create mode 100644 regression-test/suites/javaudf_p0/test_javaudf_string.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
