FrankChen021 commented on code in PR #19718:
URL: https://github.com/apache/druid/pull/19718#discussion_r3630301654
##########
sql/src/main/java/org/apache/druid/sql/calcite/schema/SystemSchema.java:
##########
@@ -517,6 +536,104 @@ private Iterator<AvailableSegmentMetadata>
getAuthorizedAvailableSegments(
return authorizedSegments.iterator();
}
+ /**
+ * Best-effort extraction of an exact-match {@code datasource} constraint
(column
+ * {@link #DATASOURCE_COLUMN}) from the pushed-down filters, so
sys.segments can restrict its scan
+ * to the matching datasources rather than materializing every segment in
the cluster. Handles
+ * {@code datasource = 'x'}, {@code datasource IN (...)} (normalized by
Calcite to SEARCH) and
+ * OR-of-equalities. Returns {@code null} when no usable datasource
predicate is present, in which
+ * case the previous full-scan behavior is retained. Because the filters
are not removed from the
+ * planner's filter list, Calcite still applies them and correctness holds
even if this extraction
+ * is conservative or over-broad.
+ */
+ @Nullable
+ static Set<String> getDataSourceFilter(List<RexNode> filters)
+ {
+ Set<String> result = null;
+ for (RexNode conjunct : filters) {
+ final Set<String> values = extractDataSourceValues(conjunct);
+ if (values != null) {
+ // Filters are implicitly ANDed, so intersect the datasource sets
they each constrain.
+ result = (result == null) ? values : Sets.intersection(result,
values).immutableCopy();
+ }
+ }
+ return result;
+ }
+
+ @Nullable
+ private static Set<String> extractDataSourceValues(RexNode node)
+ {
+ if (!(node instanceof RexCall)) {
+ return null;
+ }
+ final RexCall call = (RexCall) node;
+ switch (call.getKind()) {
Review Comment:
[P1] Handle compound predicates before scanning
Calcite's FILTER_SCAN rule can pass a normal SQL conjunction as a single
`AND(...)` RexCall, but this switch only handles EQUALS, OR, and SEARCH. As a
result, a common query such as `WHERE datasource = 'abc' AND is_active = 1`
returns no datasource constraint and both metadata sources still perform a
full-cluster scan, defeating the main purpose of this change. Please handle AND
by intersecting its extractable conjuncts and add a compound-filter regression
test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
