HappenLee commented on code in PR #65837:
URL: https://github.com/apache/doris/pull/65837#discussion_r3780875501
##########
fe/fe-core/src/main/java/org/apache/doris/planner/OlapScanNode.java:
##########
@@ -1420,6 +1423,43 @@ void
setPartitionBoundariesForRuntimeFilter(TOlapScanNode olapScanNode) {
}
}
+ private boolean hasRfDrivingBucketPruning() {
+ PlanNodeId myId = this.getId();
+ for (RuntimeFilter rf : runtimeFilters) {
+ if (rf.canPruneBucketsFor(myId)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ public void setRuntimeFilterBucketPruneParameters() {
+ if (!hasRfDrivingBucketPruning()) {
+ return;
+ }
+ for (TScanRangeLocations locations : scanRangeLocations) {
+ TPaloScanRange scanRange =
locations.getScanRange().getPaloScanRange();
+ Long bucketInfo = tabletId2BucketInfo.get(scanRange.getTabletId());
+ Preconditions.checkState(bucketInfo != null &&
decodeBucketNum(bucketInfo) > 0,
+ "missing bucket metadata for runtime-filter bucket
pruning, tablet=%s",
+ scanRange.getTabletId());
Review Comment:
已在 11684adcaa2 修复。移除了 Preconditions.checkState。OlapScanNode 现在先验证全部 scan
range；任一 tablet 的 bucket 元数据缺失或非法时，只打印一次 warning，并对整个 scan node 关闭 bucket
pruning 后继续规划。第二遍才统一写入 bucket 字段，因此也满足 BE 对同一 local state 内 range 必须全部携带或全部不携带
bucket 元数据的协议。新增 debug point 强制一个 tablet 缺失元数据，并用两个 range 的单测确认规划继续且两个 range
都不会写入 bucket 字段。
##########
fe/fe-core/src/main/java/org/apache/doris/planner/OlapScanNode.java:
##########
@@ -1420,6 +1423,43 @@ void
setPartitionBoundariesForRuntimeFilter(TOlapScanNode olapScanNode) {
}
}
+ private boolean hasRfDrivingBucketPruning() {
+ PlanNodeId myId = this.getId();
+ for (RuntimeFilter rf : runtimeFilters) {
+ if (rf.canPruneBucketsFor(myId)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ public void setRuntimeFilterBucketPruneParameters() {
Review Comment:
已在 11684adcaa2 修复。删除了 ThriftPlansBuilder 中的全局预处理，改为由 OlapScanNode.toThrift()
直接附加 bucket 元数据。方法使用 plan-scoped synchronized 幂等标记：第一个 worker 遍历并写入共享
TScanRange，后续 worker 直接返回，不会重复扫描全部 tablets。对应多 worker 幂等单测已覆盖。
##########
fe/fe-core/src/main/java/org/apache/doris/qe/runtime/ThriftPlansBuilder.java:
##########
@@ -205,6 +208,19 @@ static void setRuntimePredicateIfNeed(Collection<ScanNode>
scanNodes) {
}
}
+ static void setRuntimeFilterBucketPruneParametersIfNeeded(
+ Collection<ScanNode> scanNodes, ConnectContext connectContext) {
+ if (connectContext == null
+ ||
!connectContext.getSessionVariable().isEnableRuntimeFilterBucketPrune()) {
+ return;
Review Comment:
已在 11684adcaa2 修复。enable_runtime_filter_bucket_prune 和
enable_runtime_filter_partition_prune 现在只在 Nereids runtime filter 落到最终 scan
target、生成 prune metadata 时统一读取一次；ThriftPlansBuilder、legacy
RuntimeFilter.toThrift 和 OlapScanNode 的后续序列化路径不再重复判断。PhysicalPlanTranslatorTest
会在 metadata 生成后切换 session variable，验证序列化结果不再受后续开关变化影响。
##########
fe/fe-core/src/main/java/org/apache/doris/planner/RuntimeFilter.java:
##########
@@ -391,6 +392,14 @@ public TRuntimeFilterDesc toThrift() {
}
}
+ boolean enableRfBucketPrune = rfPruneCtx != null
+ &&
rfPruneCtx.getSessionVariable().isEnableRuntimeFilterBucketPrune();
Review Comment:
已在 11684adcaa2 修复。legacy planner RuntimeFilter 的 toThrift 不再读取
partition/bucket pruning 的 session variable，只序列化上游已经生成并冻结的 metadata。判断进一步前移到了
Nereids runtime filter 的生成位置，而不是留在 translator 中。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
