Gabriel39 commented on code in PR #66685:
URL: https://github.com/apache/doris/pull/66685#discussion_r3780453739
##########
be/src/agent/be_exec_version_manager.cpp:
##########
@@ -126,8 +126,10 @@ void
BeExecVersionManager::check_function_compatibility(int current_be_exec_vers
// a. use new fixed object serialization way.
// 11: start from master
// a. enforce Iceberg SQL MERGE cardinality only when every executing BE
supports it.
+// 12: start from Doris 4.1
Review Comment:
[P1] Allocate execution version 12 on master first\n\nThis PR targets
`branch-4.1` while current master still defines the maximum execution version
as 11, despite the invariant directly above this code. Assigning meaning 12
only in a release branch forks the protocol namespace: a later master build
either rejects version-12 FE requests or may assign version 12 a different
meaning, breaking cross-release rolling upgrades. Please land/reserve the same
version semantics on master before backporting it, or avoid consuming a new BE
execution version here.
##########
be/src/exec/operator/exchange_sink_operator.cpp:
##########
@@ -155,28 +156,32 @@ Status ExchangeSinkLocalState::init(RuntimeState* state,
LocalSinkStateInfo& inf
p._tablet_sink_partition, p._tablet_sink_location,
p._tablet_sink_tuple_id, this);
RETURN_IF_ERROR(_partitioner->init({}));
RETURN_IF_ERROR(_partitioner->prepare(state, {}));
- } else if (_part_type == TPartitionType::HIVE_TABLE_SINK_HASH_PARTITIONED)
{
- _partition_count =
- channels.size() *
config::table_sink_partition_write_max_partition_nums_per_writer;
- _partitioner = std::make_unique<ScaleWriterPartitioner>(
- channels.size(), _partition_count, channels.size(), 1,
-
config::table_sink_partition_write_min_partition_data_processed_rebalance_threshold
/
- state->task_num() ==
- 0
- ?
config::table_sink_partition_write_min_partition_data_processed_rebalance_threshold
- :
config::table_sink_partition_write_min_partition_data_processed_rebalance_threshold
/
- state->task_num(),
-
config::table_sink_partition_write_min_data_processed_rebalance_threshold /
- state->task_num() ==
- 0
- ?
config::table_sink_partition_write_min_data_processed_rebalance_threshold
- :
config::table_sink_partition_write_min_data_processed_rebalance_threshold /
- state->task_num());
-
+ } else if (_part_type ==
TPartitionType::EXTERNAL_TABLE_SINK_HASH_PARTITIONED) {
+ if (state->be_exec_version() <
SUPPORT_EXTERNAL_TABLE_SINK_HASH_VERSION) {
Review Comment:
[P1] Preserve legacy value-7 routing for pre-v12 FEs\n\n`TPartitionType`
value 7 was already `HIVE_TABLE_SINK_HASH_PARTITIONED`. During a BE-first
rolling upgrade, an old FE sends value 7 with `be_exec_version=11` and no new
metadata; the upgraded BE enters this branch and rejects the request before it
can use the legacy ScaleWriter semantics. Consequently, partitioned
Hive/Iceberg/MaxCompute writes that land on an upgraded BE fail throughout the
upgrade window. Please retain the legacy direct-hash/ScaleWriter path for
pre-v12 requests (or introduce and gate a distinct wire value), and require the
new metadata only for v12 plans.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
