Gabriel39 commented on code in PR #66685:
URL: https://github.com/apache/doris/pull/66685#discussion_r3780453735
##########
be/src/exec/operator/exchange_sink_operator.cpp:
##########
@@ -155,28 +156,32 @@ Status ExchangeSinkLocalState::init(RuntimeState* state,
LocalSinkStateInfo& inf
p._tablet_sink_partition, p._tablet_sink_location,
p._tablet_sink_tuple_id, this);
RETURN_IF_ERROR(_partitioner->init({}));
RETURN_IF_ERROR(_partitioner->prepare(state, {}));
- } else if (_part_type == TPartitionType::HIVE_TABLE_SINK_HASH_PARTITIONED)
{
- _partition_count =
- channels.size() *
config::table_sink_partition_write_max_partition_nums_per_writer;
- _partitioner = std::make_unique<ScaleWriterPartitioner>(
- channels.size(), _partition_count, channels.size(), 1,
-
config::table_sink_partition_write_min_partition_data_processed_rebalance_threshold
/
- state->task_num() ==
- 0
- ?
config::table_sink_partition_write_min_partition_data_processed_rebalance_threshold
- :
config::table_sink_partition_write_min_partition_data_processed_rebalance_threshold
/
- state->task_num(),
-
config::table_sink_partition_write_min_data_processed_rebalance_threshold /
- state->task_num() ==
- 0
- ?
config::table_sink_partition_write_min_data_processed_rebalance_threshold
- :
config::table_sink_partition_write_min_data_processed_rebalance_threshold /
- state->task_num());
-
+ } else if (_part_type ==
TPartitionType::EXTERNAL_TABLE_SINK_HASH_PARTITIONED) {
+ if (state->be_exec_version() <
SUPPORT_EXTERNAL_TABLE_SINK_HASH_VERSION) {
Review Comment:
[P1] Preserve legacy value-7 routing for pre-v12 FEs
`TPartitionType` value 7 was already `HIVE_TABLE_SINK_HASH_PARTITIONED`.
During a BE-first rolling upgrade, an old FE sends value 7 with
`be_exec_version=11` and no new metadata; the upgraded BE enters this branch
and rejects the request before it can use the legacy ScaleWriter semantics.
Consequently, partitioned Hive/Iceberg/MaxCompute writes that land on an
upgraded BE fail throughout the upgrade window. Please retain the legacy
direct-hash/ScaleWriter path for pre-v12 requests (or introduce and gate a
distinct wire value), and require the new metadata only for v12 plans.
##########
be/src/agent/be_exec_version_manager.cpp:
##########
@@ -126,8 +126,10 @@ void
BeExecVersionManager::check_function_compatibility(int current_be_exec_vers
// a. use new fixed object serialization way.
// 11: start from master
// a. enforce Iceberg SQL MERGE cardinality only when every executing BE
supports it.
+// 12: start from Doris 4.1
Review Comment:
[P1] Allocate execution version 12 on master first
This PR targets `branch-4.1` while current master still defines the maximum
execution version as 11, despite the invariant directly above this code.
Assigning meaning 12 only in a release branch forks the protocol namespace: a
later master build either rejects version-12 FE requests or may assign version
12 a different meaning, breaking cross-release rolling upgrades. Please
land/reserve the same version semantics on master before backporting it, or
avoid consuming a new BE execution version here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
