Hi, Shekhar
> 1. Optimizer does not use both - ProjectableTableSource and > FilterableTableSource - in a single query even if the source implements both > interfaces. Each interface works correctly if implemented independently. I didn’t see your custom source implementation, but I think the two interfaces should work together, you can refer the ParquetTableSource and OrcTableSource implementation details. You can also try ShengKai's proposal to use the new interface if you’re willing to upgrade to Flink 1.12, the source FilesystemTableSource implements SupportsProjectionPushDown and SupportsFilterPushDown interfaces. > 2. Implementations of FilterableTableSource fail inside the optimizer for a > few TPC-DS queries in batch mode. IIRC, all TPC-DS tests can pass using Parquet, which means the two interfaces should work fine? > > 3. And finally, filter expressions containing the current timestamp (& now) > function are not resolved to constant values during predicate pushdown > optimizer. Let's take the following SQL query for example - select count(*) > from T0 where T0.C2 >= current_timestamp. Here, applyPredicate method of > FilterableTableSource receives predicate as a CallExpression of form > greaterThanOrEqual(C2, currentTimestamp()). I'd have expected > currentTimestamp to be resolved to a constant value that is identitcal across > all usages of currentTimestamp in the query. The CURRENT_TIMESTAMP & NOW & CURRENT_DATE & CURRENT_TIME functions in Flink batch/stream sql have same behavior that calculate the function value for per record, not like hive , which calculate the function value once before the query execution and thus it can be treated as a constant value. To be honest, I tend to use a constant value for the function in batch，which is more intuitive. Could you help create an issue for this, thus we can have more discussion under the issue. Best, Leonard
