Hi Josh, Does your TableSource also implement ProjectableTableSource? If yes, you need to make sure that the filter information is also forwarded if ProjectableTableSource.projectFields() is called after FilterableTableSource.applyPredicate(). Also make sure to correctly implement FilterableTableSource.isFilterPushedDown().
Hope this helps, Fabian Am Di., 30. Apr. 2019 um 22:29 Uhr schrieb Josh Bradt < josh.br...@klaviyo.com>: > Hi all, > > I'm trying to implement filter push-down on a custom BatchTableSource that > retrieves data from a REST API and returns it as POJO instances. I've > implemented FilterableTableSource as described in the docs, returning a new > instance of my table source containing the predicates that I've removed > from the list of predicates passed into applyPredicate. However, when > getDataSet is eventually called, it's called on the instance of the table > source that was originally registered with the table environment, which > does not have any filters in it. I've stepped through the code in a > debugger, and applyPredicates is definitely being called, and it's > definitely returning new instances of my table source, but they don't seem > to be being used. > > I also played with the OrcTableSource, which is the only example of a > push-down filter implementation I could find, and it doesn't behave this > way. When I set a breakpoint in getDataSet in that case, it's being called > on one of the new instances of the table source that contains the accepted > filters. > > Are there any other requirements for implementing push-down filters that > aren't listed in the docs? Or does anyone have any tips for this? > > Thanks, > > Josh > > -- > Josh Bradt > Software Engineer > 225 Franklin St, Boston, MA 02110 > klaviyo.com <https://www.klaviyo.com> > [image: Klaviyo Logo] >
