Re: Iceberg connector

Hi Chetas, > the only way out to use only the DataStream API (and not the table api) if I want to use a custom splitComparator? You can use watermark generation, and with that, watermark based split ordering using the table api. OTOH, currently there is no way to define a custom comparator using

Re: Iceberg connector

Hi Péter, Great! Thanks! The resources are really useful. I don't have TABLE_EXEC_ICEBERG_USE_FLIP27_SOURCE set so it is the FlinkSource

Re: Iceberg connector

Hi Chetas, See my answers below: On Tue, Apr 16, 2024, 06:39 Chetas Joshi wrote: > Hello, > > I am running a batch flink job to read an iceberg table. I want to > understand a few things. > > 1. How does the FlinkSplitPlanner decide which fileScanTasks (I think one > task corresponds to one da

Iceberg connector

Hello, I am running a batch flink job to read an iceberg table. I want to understand a few things. 1. How does the FlinkSplitPlanner decide which fileScanTasks (I think one task corresponds to one data file) need to be clubbed together within a single split and when to create a new split? 2. Whe