Re: Filter push-down not working for a custom BatchTableSource

2019-05-04 Thread Rong Rong
Hi Josh, I think I found the root cause of this issue (please see my comment in https://issues.apache.org/jira/browse/FLINK-12399). As of now, you can try override the expalinSource() interface to let calcite know that the tablesource after calling applyPredicate is different from the one before c

Re: Filter push-down not working for a custom BatchTableSource

2019-05-03 Thread Josh Bradt
Hi Fabian, Thanks for taking a look. I've filed this ticket: https://issues.apache.org/jira/browse/FLINK-12399 Thanks, Josh On Fri, May 3, 2019 at 3:41 AM Fabian Hueske wrote: > Hi Josh, > > The code looks good to me. > This seems to be a bug then. > It's strange that it works for ORC. > > Wo

Re: Filter push-down not working for a custom BatchTableSource

2019-05-03 Thread Fabian Hueske
Hi Josh, The code looks good to me. This seems to be a bug then. It's strange that it works for ORC. Would you mind opening a Jira ticket and maybe a simple reproducable code example? Thank you, Fabian Am Do., 2. Mai 2019 um 18:23 Uhr schrieb Josh Bradt : > Hi Fabian, > > Thanks for your reply

Re: Filter push-down not working for a custom BatchTableSource

2019-05-02 Thread Josh Bradt
Hi Fabian, Thanks for your reply. My custom table source does not implement ProjectableTableSource. I believe that isFilterPushedDown is implemented correctly since it's nearly identical to what's written in the OrcTableSource. I pasted a slightly simplified version of the implementation below. If

Re: Filter push-down not working for a custom BatchTableSource

2019-05-02 Thread Fabian Hueske
Hi Josh, Does your TableSource also implement ProjectableTableSource? If yes, you need to make sure that the filter information is also forwarded if ProjectableTableSource.projectFields() is called after FilterableTableSource.applyPredicate(). Also make sure to correctly implement FilterableTableS

Filter push-down not working for a custom BatchTableSource

2019-04-30 Thread Josh Bradt
Hi all, I'm trying to implement filter push-down on a custom BatchTableSource that retrieves data from a REST API and returns it as POJO instances. I've implemented FilterableTableSource as described in the docs, returning a new instance of my table source containing the predicates that I've remov