[DISCUSS] DataFilter pushed down dynamically

2022-07-12 Thread qcsd2011
Hi dev,We would like raise a discussion about supporting `DataFilter pushed down dynamically`. We are trying to push down dataFilter with PartitionFilter, and prune partitionFilter at runtime before push to parquet, which can push less filter to parquet. More details can be found in 3

[DISCUSS] DataFilter pushed down dynamically

2022-07-12 Thread qcsd2011
Hi dev,We would like raise a discussion about supporting `DataFilter pushed down dynamically`. We are trying to push down dataFilter with PartitionFilter, and prune partitionFilter at runtime before push to parquet, which can push less filter to parquet. More details can be found in 3

[DISCUSS] DataFilter pushed down dynamically

2022-07-12 Thread stc
Hi dev, We would like raise a discussion about supporting `DataFilter pushed down dynamically`. We are trying to push down dataFilter with PartitionFilter, and prune partitionFilter at runtime before push to parquet, which can push less filter to parquet. More details can be found in 35669.

[DISCUSS] DataFilter pushed down dynamically

2022-07-12 Thread Jacky Lee
Hi dev, We would like to raise a discussion about supporting `DataFilter pushed down dynamically`. We are trying to push down dataFilter with PartitionFilter, and prune partitionFilter at runtime before push to parquet, which can push less filter to parquet. More details can be found in 35669

[DISCUSS] DataFilter pushed down dynamically

2022-07-12 Thread Jacky Lee
Hi dev, We would like to raise a discussion about supporting `DataFilter pushed down dynamically`. We are trying to push down dataFilter with PartitionFilter, and prune partitionFilter at runtime before push to parquet, which can push less filter to parquet. More details can be found in 35669

[DISCUSS] DataFilter pushed down dynamically

2022-07-12 Thread qcsd2011
Hi dev,We would like to raise a discussion about supporting `DataFilterpushed down dynamically`. We are trying to push down dataFilter withPartitionFilter, and prune partitionFilter at runtime before push toparquet, which can push less filter to parquet. More details can

[DISCUSS] DataFilter pushed down dynamically

2022-07-12 Thread Jacky Lee
Hi dev, We would like raise a discussion about supporting `DataFilter pushed down dynamically`. We are trying to push down dataFilter with PartitionFilter, and prune partitionFilter at runtime before push to parquet, which can push less filter to parquet. More details can be found in 35669

[DISCUSS] DataFilter pushed down dynamically

2022-07-12 Thread Jacky Lee
Hi dev, We would like to raise a discussion about supporting `DataFilter pushed down dynamically`. We are trying to push down dataFilter with PartitionFilter, and prune partitionFilter at runtime before push to parquet, which can push less filter to parquet. More details can be found in 35669. be