This is an automated email from the ASF dual-hosted git repository.
lijibing pushed a change to branch high-priority-column
in repository https://gitbox.apache.org/repos/asf/doris.git
discard 65a71e86856 Support column level health value. (#31794)
omit 1e71f334bdf High priority queue and map. (#31509)
omit eafbcb51507 Collect high priority columns. (#31235)
add d8b4edbd60a [bugfix](wgcore) map at only get reference and it will
core in multithread (#31702)
add b71686dfb28 [pipelineX](bug) Fix incorrect join operator judgement
(#31690)
add d459ac239f0 [chore](ci) prepare jdk17 in cloud_p* and performance
(#31685)
add 1ef6a1bbc55 [fix](jdbc catalog) Change Druid Pool dependency to
version 1.2.11 (#31703)
add a970fef2517 [refactor](jdbc catalog) split sqlserver jdbc executor
(#31679)
add 5048c6ec6a4 [feature](Nereids) parallel output file (#31623)
add 3e910206991 [fix](testcases) make all auto partition P2 cases
nonConcurrent (#31708)
add c90728cd6af fix in stats estimation (#31684)
add fb2c4011e4f [fix](scan) Fix missing sync rowsets in cloud mode (#31717)
add be418110cb4 [enhancement](nereids)support null partition for list
partition (#31613)
add 183ff65f2e3 [Feature](topn) support multiple topn filter on backend
(#31665)
add 237929d0cbc [Bug](resource) fix npe on Resource read from json (#31723)
add 28b15b2521e [fix](connections) fix connection hang after too many
connections (#31594)
add 966d500ed9e [fix](merge-cloud) fix mac compile error (#31732)
add 18f98710e14 [fix](fe) Fix NPE in FrontendServiceImpl.loadTxnCommit if
table is dropped (#31715)
add 1d6e2f96768 [fix](feut) fix TabletRepairAndBalanceTest (#31653)
add 7d55e16df5e [fix](partial update) compaction may cause update failue
(#31551)
add f73ea301534 [Enhencement](env) Checking Master branch must use JDK17
(#31587)
add b6696da4ca6 [fix](merge-cloud) fix full compaction core (#31745)
add 98671ffcf7e [fix](internal table) fix internal replica num in cloud
(#31740)
add 08bc414a20d [fix](cloud) Fix regression case `test_nereids_row_policy`
run in cloud mode (#31726)
add a47d15f5e65 [fix](test) Fix
cloud_p0/catalog_recycle_bin_p0/show.groovy failed (#31727)
add dae59ad7fa9 [bug](scan) Fix missing sync rowsets in cloud mode (#31756)
add da5b6d38664 [feature](insert)add hive table sink definition (#31662)
add ea027f3ad31 [Performance](exec) Support runtime filter in <=> join
(#31754)
add b2887367b2d [feature](function) round function defaults to rounding
normally
add 0e4191d9f44 [docs](kafka-connector) Add doris-kafka-connector usage
documentation (#31729)
add 28678ca68ee [Fix](partial update) Fix partial update load false when
schema includes auto increment column (#31725)
add 12305ce4c24 [fix](broker-load) fix broker load statement type
conversion failure (#31746)
add 1c60d29206b [opt](doc) update release verify (#31773)
add 7fbace480e3 [fix](cast) wrong result while cast const to double then
to string (#31657)
add 207aba4d0b4 Refactor active queries (#31742)
add f71b531ead9 fix](Nereids): support max_value in range partition
(#31721)
add 4a3e0bdb135 [fix](jdbc catalog) fix close abort in sqlserver (#31718)
add 2a77d8b6f38 [feature](doris compose) doris compose use jdk 17 image
(#31775)
add c1fae97b3e8 [chore](ci) remove required of pipelinex (#31791)
add 2bbbdae6a2e [fix](hive) support partition prune for
_HIVE_DEFAULT_PARTITION_ (#31736)
add 021cbfbe0e4 [chore](Nereids) change consumer rules name (#31777)
new 0f97217b72b Collect high priority columns. (#31235)
new 2a0b3add07d High priority queue and map. (#31509)
new a2c22c63e63 Support column level health value. (#31794)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (65a71e86856)
\
N -- N -- N refs/heads/high-priority-column (a2c22c63e63)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.asf.yaml | 1 -
be/src/cloud/cloud_base_compaction.cpp | 4 +
be/src/cloud/cloud_cumulative_compaction.cpp | 2 +
be/src/cloud/cloud_full_compaction.cpp | 8 +-
be/src/cloud/cloud_meta_mgr.cpp | 13 +-
be/src/common/sync_point.h | 4 +-
be/src/exec/tablet_info.cpp | 12 +
be/src/exec/tablet_info.h | 2 +
be/src/exprs/bloom_filter_func.h | 11 +-
be/src/exprs/runtime_filter.cpp | 5 +
be/src/exprs/runtime_filter.h | 1 +
be/src/olap/base_tablet.cpp | 12 +-
be/src/olap/base_tablet.h | 4 +-
be/src/olap/delta_writer_v2.cpp | 6 +-
be/src/olap/iterators.h | 1 +
be/src/olap/parallel_scanner_builder.cpp | 1 +
be/src/olap/partial_update_info.h | 3 +-
be/src/olap/rowset/beta_rowset_reader.cpp | 1 +
be/src/olap/rowset/rowset_reader_context.h | 1 +
be/src/olap/rowset/segment_v2/segment.cpp | 32 +-
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 43 +--
be/src/olap/rowset/segment_v2/segment_writer.cpp | 47 ++-
be/src/olap/rowset/segment_v2/segment_writer.h | 4 +-
.../rowset/segment_v2/vertical_segment_writer.cpp | 45 ++-
.../rowset/segment_v2/vertical_segment_writer.h | 3 +
be/src/olap/rowset_builder.cpp | 4 +
be/src/olap/tablet_meta.cpp | 1 +
be/src/olap/tablet_reader.cpp | 9 +-
be/src/olap/tablet_reader.h | 1 +
be/src/olap/tablet_schema.cpp | 1 +
be/src/olap/tablet_schema.h | 11 +
be/src/pipeline/exec/olap_scan_operator.cpp | 38 ++-
be/src/pipeline/exec/sort_sink_operator.cpp | 6 +-
be/src/pipeline/pipeline_fragment_context.cpp | 15 +-
.../pipeline_x/pipeline_x_fragment_context.cpp | 13 +-
be/src/runtime/fragment_mgr.cpp | 5 +-
be/src/runtime/query_context.h | 16 +-
be/src/runtime/runtime_state.cpp | 3 +
be/src/runtime/task_group/task_group_manager.cpp | 18 +-
be/src/vec/exec/scan/new_olap_scanner.cpp | 6 +
be/src/vec/exec/scan/vscan_node.cpp | 4 +-
be/src/vec/exec/vsort_node.cpp | 6 +-
be/src/vec/functions/round.h | 92 ++----
be/src/vec/olap/vcollect_iterator.cpp | 4 +-
bin/start_be.sh | 13 +-
bin/start_fe.sh | 12 +-
conf/fe.conf | 2 +-
docker/runtime/doris-compose/Dockerfile | 32 +-
docker/runtime/doris-compose/command.py | 28 +-
docker/runtime/doris-compose/utils.py | 28 +-
.../community/release-and-verify/release-verify.md | 4 +-
docs/en/docs/ecosystem/doris-kafka-connector.md | 192 +++++++++++
.../sql-functions/numeric-functions/round.md | 1 +
.../table-functions/active_queries.md | 45 ++-
docs/sidebars.json | 1 +
.../community/release-and-verify/release-verify.md | 4 +-
docs/zh-CN/docs/ecosystem/doris-kafka-connector.md | 193 +++++++++++
.../sql-functions/numeric-functions/round.md | 2 +-
.../table-functions/active_queries.md | 45 ++-
env.sh | 86 ++++-
.../org/apache/doris/jdbc/BaseJdbcExecutor.java | 13 +
.../org/apache/doris/jdbc/JdbcExecutorFactory.java | 2 +
.../org/apache/doris/jdbc/MySQLJdbcExecutor.java | 9 -
.../apache/doris/jdbc/SQLServerJdbcExecutor.java | 120 +++++++
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 2 +-
.../org/apache/doris/analysis/FloatLiteral.java | 5 +
.../org/apache/doris/analysis/OutFileClause.java | 2 +-
.../org/apache/doris/analysis/PartitionDesc.java | 3 -
.../org/apache/doris/analysis/PartitionValue.java | 21 +-
.../java/org/apache/doris/analysis/SlotRef.java | 2 +-
.../java/org/apache/doris/catalog/EsResource.java | 4 +
.../java/org/apache/doris/catalog/HMSResource.java | 4 +
.../org/apache/doris/catalog/HdfsResource.java | 4 +
.../doris/catalog/InternalSchemaInitializer.java | 3 +
.../apache/doris/catalog/ListPartitionItem.java | 11 -
.../apache/doris/catalog/OdbcCatalogResource.java | 1 -
.../org/apache/doris/catalog/PartitionKey.java | 139 ++++----
.../java/org/apache/doris/catalog/Resource.java | 3 +
.../java/org/apache/doris/catalog/S3Resource.java | 7 +-
.../org/apache/doris/catalog/SparkResource.java | 5 +
.../transaction/CloudGlobalTransactionMgr.java | 2 +-
.../apache/doris/common/util/PropertyAnalyzer.java | 2 +-
.../doris/datasource/hive/HiveMetaStoreCache.java | 2 -
.../org/apache/doris/load/loadv2/BulkLoadJob.java | 24 +-
.../org/apache/doris/mysql/AcceptListener.java | 107 ++++---
.../java/org/apache/doris/mysql/MysqlServer.java | 2 +-
.../glue/translator/PhysicalPlanTranslator.java | 73 ++++-
.../glue/translator/PlanTranslatorContext.java | 2 -
.../doris/nereids/parser/LogicalPlanBuilder.java | 2 +
.../pre/TurnOffPageCacheForInsertIntoSelect.java | 7 +
.../nereids/properties/RequestPropertyDeriver.java | 2 +-
.../org/apache/doris/nereids/rules/RuleSet.java | 2 +
.../org/apache/doris/nereids/rules/RuleType.java | 8 +-
.../doris/nereids/rules/analysis/BindSink.java | 10 +-
.../rules/HiveDefaultPartitionEvaluator.java | 68 ----
.../rules/expression/rules/PartitionPruner.java | 9 +-
...ogicalHiveTableSinkToPhysicalHiveTableSink.java | 49 +++
.../rules/rewrite/CollectFilterAboveConsumer.java | 2 +-
.../rules/rewrite/CollectProjectAboveConsumer.java | 4 +-
.../doris/nereids/stats/FilterEstimation.java | 2 +-
.../nereids/trees/expressions/literal/Literal.java | 2 +
.../apache/doris/nereids/trees/plans/PlanType.java | 7 +-
.../plans/commands/UpdateMvByPartitionCommand.java | 22 +-
.../trees/plans/commands/info/CreateTableInfo.java | 3 -
.../plans/commands/info/PartitionDefinition.java | 2 +-
.../commands/insert/AbstractInsertExecutor.java | 4 +-
.../plans/commands/insert/HiveInsertExecutor.java | 100 ++++++
.../commands/insert/InsertIntoTableCommand.java | 5 +
.../trees/plans/commands/insert/InsertUtils.java | 9 +-
.../trees/plans/logical/LogicalHiveTableSink.java | 160 ++++++++++
.../trees/plans/physical/PhysicalFileSink.java | 17 +
.../plans/physical/PhysicalHiveTableSink.java | 119 +++++++
.../nereids/trees/plans/visitor/SinkVisitor.java | 10 +
.../java/org/apache/doris/planner/DataSink.java | 7 +-
.../org/apache/doris/planner/HashJoinNode.java | 24 +-
.../org/apache/doris/planner/HiveTableSink.java | 67 ++++
.../doris/planner/MultiCastPlanFragment.java | 1 -
.../org/apache/doris/planner/OlapTableSink.java | 14 +-
.../org/apache/doris/planner/OriginalPlanner.java | 39 +--
.../org/apache/doris/planner/PlanFragment.java | 10 +-
.../java/org/apache/doris/planner/PlanNode.java | 4 +
.../org/apache/doris/planner/ResultFileSink.java | 45 +++
.../java/org/apache/doris/planner/ScanNode.java | 2 +-
.../main/java/org/apache/doris/qe/Coordinator.java | 4 +
.../java/org/apache/doris/qe/StmtExecutor.java | 5 +-
.../apache/doris/service/FrontendServiceImpl.java | 16 +-
.../ActiveQueriesTableValuedFunction.java | 9 -
.../doris/tablefunction/MetadataGenerator.java | 54 ++--
.../apache/doris/analysis/StmtRewriterTest.java | 12 +-
.../doris/clone/TabletRepairAndBalanceTest.java | 2 +-
.../commands/UpdateMvByPartitionCommandTest.java | 56 ++++
.../doris/planner/TableFunctionPlanTest.java | 6 +-
.../org/apache/doris/qe/OlapQueryCacheTest.java | 12 +-
fe/pom.xml | 2 +-
gensrc/proto/descriptors.proto | 1 +
gensrc/proto/olap_file.proto | 1 +
gensrc/thrift/DataSinks.thrift | 63 +++-
gensrc/thrift/Descriptors.thrift | 3 +-
gensrc/thrift/PaloInternalService.thrift | 1 +
gensrc/thrift/PlanNodes.thrift | 4 +
.../data/correctness/test_float_round_up.out | 13 +
.../unique/auto_inc_partial_update3.csv | 5 +
.../unique/test_unique_table_auto_inc.out | 16 +
.../fault_injection_p0/concurrency_update2.csv | 5 +
.../fault_injection_p0/concurrency_update3.csv | 5 +
..._partial_update_publish_conflict_with_error.out | 93 +++---
.../data/nereids_function_p0/scalar_function/D.out | 4 +-
.../data/nereids_function_p0/scalar_function/R.out | 4 +-
.../cast_function/test_cast_function.out | 3 +
.../nereids_syntax_p0/list_partition_with_null.out | 15 +
regression-test/data/query_p0/join/test_join.out | 351 +++++++++++++++++++++
.../cast_function/test_cast_function.out | 3 +
.../sql_functions/math_functions/test_round.out | 24 +-
.../issuesWithTheMostComments3.out | 2 +-
.../sql/issuesWithTheMostComments3.out | 2 +-
.../cloud_p0/conf/regression-conf-custom.groovy | 2 +-
regression-test/pipeline/cloud_p0/deploy.sh | 2 +
regression-test/pipeline/cloud_p0/prepare.sh | 5 +-
regression-test/pipeline/cloud_p0/run.sh | 4 +-
regression-test/pipeline/common/doris-utils.sh | 28 +-
regression-test/pipeline/performance/compile.sh | 6 +-
regression-test/pipeline/performance/prepare.sh | 15 +-
.../account_p0/test_nereids_row_policy.groovy | 20 ++
.../suites/correctness/test_float_round_up.groovy | 35 +-
.../unique/test_unique_table_auto_inc.groovy | 15 +-
.../suites/demo_p0/explain_action.groovy | 2 +-
.../suites/export_p0/test_outfile.groovy | 5 +-
.../hive/test_hive_default_partition.groovy | 34 +-
...rtial_update_publish_conflict_with_error.groovy | 4 +-
.../suites/nereids_p0/outfile/test_outfile.groovy | 5 +-
.../cast_function/test_cast_function.groovy | 1 +
.../list_partition_with_null.groovy | 71 +++++
.../test_list_partition_datatype.groovy | 19 --
.../diff_data/stress_test_diff_date_list.groovy | 2 +-
.../stress_test_high_concurrency_load.groovy | 2 +-
.../same_data/stress_test_same_date_range.groovy | 2 +-
.../stress_test_two_stream_load.groovy | 2 +-
.../suites/query_p0/join/test_join.groovy | 19 ++
.../cast_function/test_cast_function.groovy | 1 +
179 files changed, 2912 insertions(+), 804 deletions(-)
create mode 100644 docs/en/docs/ecosystem/doris-kafka-connector.md
create mode 100644 docs/zh-CN/docs/ecosystem/doris-kafka-connector.md
create mode 100644
fe/be-java-extensions/jdbc-scanner/src/main/java/org/apache/doris/jdbc/SQLServerJdbcExecutor.java
delete mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/HiveDefaultPartitionEvaluator.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/implementation/LogicalHiveTableSinkToPhysicalHiveTableSink.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/HiveInsertExecutor.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalHiveTableSink.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalHiveTableSink.java
create mode 100644
fe/fe-core/src/main/java/org/apache/doris/planner/HiveTableSink.java
create mode 100644
fe/fe-core/src/test/java/org/apache/doris/nereids/trees/plans/commands/UpdateMvByPartitionCommandTest.java
create mode 100644 regression-test/data/correctness/test_float_round_up.out
create mode 100644
regression-test/data/data_model_p0/unique/auto_inc_partial_update3.csv
create mode 100644
regression-test/data/nereids_syntax_p0/list_partition_with_null.out
copy
fe/be-java-extensions/jdbc-scanner/src/main/java/org/apache/doris/jdbc/JdbcExecutorFactory.java
=> regression-test/suites/correctness/test_float_round_up.groovy (58%)
create mode 100644
regression-test/suites/nereids_syntax_p0/list_partition_with_null.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
