@Arvid Heise I also do not remember exactly what were all the problems. The fact that we added some more bulk formats to the streaming file sink definitely reduced the non-supported features. In addition, the latest discussion I found on the topic was [1] and the conclusion of that discussion seems to be to remove it.
Currently, I cannot find any obvious reason why keeping the BucketingSink, apart from the fact that we do not have a migration plan unfortunately. This is why I posted this to dev@ and user@. Cheers, Kostas [1] https://lists.apache.org/thread.html/r799be74658bc7e169238cc8c1e479e961a9e85ccea19089290940ff0%40%3Cdev.flink.apache.org%3E On Wed, Oct 14, 2020 at 8:03 AM Arvid Heise <ar...@ververica.com> wrote: > > I remember this conversation popping up a few times already and I'm in > general a big fan of removing BucketingSink. > > However, until now there were a few features lacking in StreamingFileSink > that are present in BucketingSink and that are being actively used (I can't > exactly remember them now, but I can look it up if everyone else is also > suffering from bad memory). Did we manage to add them in the meantime? If > not, then it feels rushed to remove it at this point. > > On Tue, Oct 13, 2020 at 2:33 PM Kostas Kloudas <kklou...@gmail.com> wrote: > > > @Chesnay Schepler Off the top of my head, I cannot find an easy way > > to migrate from the BucketingSink to the StreamingFileSink. It may be > > possible but it will require some effort because the logic would be > > "read the old state, commit it, and start fresh with the > > StreamingFileSink." > > > > On Tue, Oct 13, 2020 at 2:09 PM Aljoscha Krettek <aljos...@apache.org> > > wrote: > > > > > > On 13.10.20 14:01, David Anderson wrote: > > > > I thought this was waiting on FLIP-46 -- Graceful Shutdown Handling -- > > and > > > > in fact, the StreamingFileSink is mentioned in that FLIP as a > > motivating > > > > use case. > > > > > > Ah yes, I see FLIP-147 as a more general replacement for FLIP-46. Thanks > > > for the reminder, we should close FLIP-46 now with an explanatory > > > message to avoid confusion. > > > > > -- > > Arvid Heise | Senior Java Developer > > <https://www.ververica.com/> > > Follow us @VervericaData > > -- > > Join Flink Forward <https://flink-forward.org/> - The Apache Flink > Conference > > Stream Processing | Event Driven | Real Time > > -- > > Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany > > -- > Ververica GmbH > Registered at Amtsgericht Charlottenburg: HRB 158244 B > Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji > (Toni) Cheng
