[
https://issues.apache.org/jira/browse/FLINK-39421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18112129#comment-18112129
]
Yuepeng Pan commented on FLINK-39421:
-------------------------------------
Hi, [~jhughes]
Flink 2.4 will go into code freeze in mid-October. If this FLIP is planned to
be included in the 2.4 release, please help get it registered on the [1] page.
Thanks.
[1] https://cwiki.apache.org/confluence/spaces/FLINK/pages/449286596/2.4+Release
> Metadata filter push-down for table sources
> -------------------------------------------
>
> Key: FLINK-39421
> URL: https://issues.apache.org/jira/browse/FLINK-39421
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Reporter: Jim Hughes
> Assignee: Jim Hughes
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.0
>
>
> Predicates on metadata columns (e.g., Kafka offset, timestamp, partition) are
> excluded from the existing SupportsFilterPushDown mechanism.
> FilterPushDownSpec serializes RexInputRef indices relative to the scan's
> produced row type, but metadata columns are appended after physical columns.
> When ProjectPushDownSpec narrows the row type during compiled plan
> restoration, metadata indices become out-of-bounds, causing
> ArrayIndexOutOfBoundsException.
> This ticket introduces a dedicated metadata filter push-down path:
> - New supportsMetadataFilterPushDown() and applyMetadataFilters() default
> methods on SupportsReadingMetadata (opt-in; existing sources unaffected)
> - New MetadataFilterPushDownSpec that snapshots the row type at optimization
> time, solving the index mismatch
> - Extension to PushFilterIntoTableSourceScanRule to classify predicates as
> physical, metadata, or mixed
> See FLIP-574
> (https://cwiki.apache.org/confluence/display/FLINK/FLIP-574+Metadata+Filter+Push-Down+for+Table+Sources)
> for full design.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
