Yaroslav Tkachenko created FLINK-30873: ------------------------------------------
Summary: FileSystem Table API connector doesn't discover new files when partitions are specified Key: FLINK-30873 URL: https://issues.apache.org/jira/browse/FLINK-30873 Project: Flink Issue Type: Bug Components: Connectors / FileSystem Affects Versions: 1.16.1 Reporter: Yaroslav Tkachenko When *source.monitor-interval* is configured I expect the source connector to discover new files. However, if the source table was created with partitions (using PARTITIONED BY), it only discovers new files in the partition folders that existed during the startup; it doesn't discover new partitions. I believe the problem is [here|https://github.com/apache/flink/blob/5f2d088a2713ced5c6ce072db92f4378f73bc739/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/FileSystemTableSource.java#L276-L286], only these file paths are monitored internally, no partition discovery is currently implemented. -- This message was sent by Atlassian Jira (v8.20.10#820010)