Thanks for the reply. That helps..
On Thu, Mar 27, 2025, 7:29 AM Wenchen Fan wrote:
> The file source in Spark has not been migrated to DS v2 yet and uses
> dedicated catalyst rules to do runtime filtering, e.g. PartitionPruning
> and PlanDynamicPruningFilters
>
> On Thu, Mar 27, 2025 at 6:53 PM
The file source in Spark has not been migrated to DS v2 yet and uses
dedicated catalyst rules to do runtime filtering, e.g. PartitionPruning
and PlanDynamicPruningFilters
On Thu, Mar 27, 2025 at 6:53 PM Asif Shahid wrote:
> Hi Experts,
> Could you please allow me to pick your brain on the follo
Hi Experts,
Could you please allow me to pick your brain on the following:
For Hive Tables ( managed), the scan operator is FileSourceScanExec.
Is there any particular reason why its underlying HadoopFSRelations'
field, FileFormat does not implement an interface like
SupportsRuntimeFiltering ?
Li
