+1 It has been deprecated for some time and the StreamingFileSink has stabalized with a large number of formats and features.
Plus, the bucketing sink only implements a small number of stable interfaces[1]. I would expect users to continue to use the bucketing sink from the 1.11 release with future versions for some time. Seth https://github.com/apache/flink/blob/2ff3b771cbb091e1f43686dd8e176cea6d435501/flink-connectors/flink-connector-filesystem/src/main/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.java#L170-L172 On Thu, Oct 15, 2020 at 2:57 PM Kostas Kloudas <kklou...@gmail.com> wrote: > @Arvid Heise I also do not remember exactly what were all the > problems. The fact that we added some more bulk formats to the > streaming file sink definitely reduced the non-supported features. In > addition, the latest discussion I found on the topic was [1] and the > conclusion of that discussion seems to be to remove it. > > Currently, I cannot find any obvious reason why keeping the > BucketingSink, apart from the fact that we do not have a migration > plan unfortunately. This is why I posted this to dev@ and user@. > > Cheers, > Kostas > > [1] > https://lists.apache.org/thread.html/r799be74658bc7e169238cc8c1e479e961a9e85ccea19089290940ff0%40%3Cdev.flink.apache.org%3E > > On Wed, Oct 14, 2020 at 8:03 AM Arvid Heise <ar...@ververica.com> wrote: > > > > I remember this conversation popping up a few times already and I'm in > > general a big fan of removing BucketingSink. > > > > However, until now there were a few features lacking in StreamingFileSink > > that are present in BucketingSink and that are being actively used (I > can't > > exactly remember them now, but I can look it up if everyone else is also > > suffering from bad memory). Did we manage to add them in the meantime? If > > not, then it feels rushed to remove it at this point. > > > > On Tue, Oct 13, 2020 at 2:33 PM Kostas Kloudas <kklou...@gmail.com> > wrote: > > > > > @Chesnay Schepler Off the top of my head, I cannot find an easy way > > > to migrate from the BucketingSink to the StreamingFileSink. It may be > > > possible but it will require some effort because the logic would be > > > "read the old state, commit it, and start fresh with the > > > StreamingFileSink." > > > > > > On Tue, Oct 13, 2020 at 2:09 PM Aljoscha Krettek <aljos...@apache.org> > > > wrote: > > > > > > > > On 13.10.20 14:01, David Anderson wrote: > > > > > I thought this was waiting on FLIP-46 -- Graceful Shutdown > Handling -- > > > and > > > > > in fact, the StreamingFileSink is mentioned in that FLIP as a > > > motivating > > > > > use case. > > > > > > > > Ah yes, I see FLIP-147 as a more general replacement for FLIP-46. > Thanks > > > > for the reminder, we should close FLIP-46 now with an explanatory > > > > message to avoid confusion. > > > > > > > > > -- > > > > Arvid Heise | Senior Java Developer > > > > <https://www.ververica.com/> > > > > Follow us @VervericaData > > > > -- > > > > Join Flink Forward <https://flink-forward.org/> - The Apache Flink > > Conference > > > > Stream Processing | Event Driven | Real Time > > > > -- > > > > Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany > > > > -- > > Ververica GmbH > > Registered at Amtsgericht Charlottenburg: HRB 158244 B > > Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji > > (Toni) Cheng >
