Re: Re: Iceberg table maintenance

2024-05-03 Thread Péter Váry
Hi Flink Team, The Iceberg table maintenance proposal is on vote on the Iceberg dev list [1]. Non-binding votes are important too, so if you are interested, please vote. Thanks, Peter [1] - https://lists.apache.org/thread/qhz17ppdbb57ql356j49qqk3nyk59rvm On Tue, Apr 2, 2024, 08:35 Péter Váry

Re: Iceberg table maintenance

2024-04-09 Thread Ahmed Hamdy
Thanks Peter for forwarding this. Best Regards Ahmed Hamdy On Tue, 9 Apr 2024 at 05:51, Péter Váry wrote: > Forwarding the invite for the discussion we plan to do with the Iceberg > folks, as some of you might be interested in this. > > -- Forwarded message - > From: Brian Olsen

Iceberg table maintenance

2024-04-08 Thread Péter Váry
Forwarding the invite for the discussion we plan to do with the Iceberg folks, as some of you might be interested in this. -- Forwarded message - From: Brian Olsen Date: Mon, Apr 8, 2024, 18:29 Subject: Re: Flink table maintenance To: Hey Iceberg nation, I would like to share

Re: Re: Iceberg table maintenance

2024-04-01 Thread Péter Váry
Thanks Wenjin for your response. See my answers below: On Tue, Apr 2, 2024, 04:08 wenjin wrote: > Hi Peter, > Thanks a lot for your answers, this detailed explanation has cleared my > confusions and been greatly beneficial to me. If you don't mind, could I > discuss two more questions with your

RE: Re: Iceberg table maintenance

2024-04-01 Thread wenjin
Hi Peter, Thanks a lot for your answers, this detailed explanation has cleared my confusions and been greatly beneficial to me. If you don't mind, could I discuss two more questions with your? As you mentioned in your proposal and answers, the maintenance tasks may cause resource interference a

Re: Iceberg table maintenance

2024-03-30 Thread Péter Váry
Hi Wenjin, See my answers below: On Sat, Mar 30, 2024, 10:54 wenjin wrote: > Hi Peter, > > I am interested in your proposal and think make iceberg Flink Connector > support running maintenance task is meaningful . If possible, could you > help me clarify a few confusions. > > - When the iceberg

RE: Iceberg table maintenance

2024-03-30 Thread wenjin
Hi Peter, I am interested in your proposal and think make iceberg Flink Connector support running maintenance task is meaningful . If possible, could you help me clarify a few confusions. - When the iceberg table is written by single Flink job (use case1, 2),the maintenance tasks will be added

Iceberg table maintenance

2024-03-28 Thread Péter Váry
Hi Team, I am working on adding a possibility to the Flink Iceberg connector to run maintenance tasks on the Iceberg tables. This will fix the small files issues and in the long run help compacting the high number of positional and equality deletes created by Flink tasks writing CDC data to Iceber