Re: [Proposal] Add support for Flink Maintenance in Iceberg

2024-05-07 Thread Zhu Zhu
+1 Thanks, Zhu Jean-Baptiste Onofré 于2024年5月7日周二 16:17写道: > +1 > > Regards > JB > > On Fri, May 3, 2024 at 8:30 PM Péter Váry > wrote: > > > > Hi everyone, > > > > I would like to make a proposal [1] to support Flink Table Maintenance > in Iceberg. The main goal is to have a solution where Fli

Re: Flink table maintenance

2024-04-19 Thread Zhu Zhu
Hi Peter, > Flink job doing a quick small file reduction after collecting several commits Triggering maintenance tasks as soon as possible is a valid point to me. But I'm not sure about the priority of it, compared to maintenance tasks which may happen with a delay of few seconds. > Single, cont

Re: Flink table maintenance

2024-04-18 Thread Zhu Zhu
Hi Peter, Thanks for starting this discussion. I'm a bit uncertain about the necessity of using extended sink topology to run maintenance tasks. Creating a separate pipeline for maintenance monitor, scheduler and tasks sounds a better choice to me in most cases. Advantages I can think of include