Re: Pushdown in DataSourceV2 question

2018-12-11 Thread Noritaka Sekiyama
guarantee >>> the correctness of the final query? >>> >>> Consider a filter pushed down to the data source, either Spark has to >>> know if it has been applied or not, or it has to re-apply the filter anyway >>> (and pay the price for that). >>&

Pushdown in DataSourceV2 question

2018-12-08 Thread Noritaka Sekiyama
2 because it would require some API level changes. Warm regards, Noritaka Sekiyama