Re: [DISCUSS] Drop Bucketing Sink

2020-03-26 Thread Robert Metzger
By the way: I have disabled the test in the Hadoop 2.4.1 build, so my original problem is resolved. I'm not convinced that posting on the user@ ml is the right approach. The last few questions to the user@ list asking for feedback have not really been answered. I believe that we have deprecated th

Re: [DISCUSS] Drop Bucketing Sink

2020-03-17 Thread Kostas Kloudas
Thanks Robert for all this, I think that we should also post a thread in the user ML so that users can also comment on the topic. What do you think? Kostas On Mon, Mar 16, 2020 at 12:27 PM Robert Metzger wrote: > > Thank you all for your feedback. > > I will try to fix the test then (or disabl

Re: [DISCUSS] Drop Bucketing Sink

2020-03-16 Thread Robert Metzger
Thank you all for your feedback. I will try to fix the test then (or disable it). Here's a ticket for dropping the BucketingSink: https://issues.apache.org/jira/browse/FLINK-16616 Please mark whatever we consider necessary as a "depends on" ticket. @David / @Seth: Where are the tickets depending

Re: [DISCUSS] Drop Bucketing Sink

2020-03-13 Thread Guowei Ma
+1 to drop it. To Jingsong : we are planning to implement the orc StreamingFileSink in 1.11. I think users also could reference the old BucktSink from the old version. Best, Guowei Jingsong Li 于2020年3月13日周五 上午10:07写道: > Hi Robert, > > +1 to drop it but maybe not 1.11. > > ORC has not been sup

Re: [DISCUSS] Drop Bucketing Sink

2020-03-13 Thread Yu Li
Thanks for bringing up this discussion Robert! According to the inputs, I suggest we create an umbrella JIRA issue to track all critical improvements StreamingFileSink should have before we could completely discard bucketing sink, so we could have a clear view of the progress and how soon we could

Re: [DISCUSS] Drop Bucketing Sink

2020-03-13 Thread jinhai wang
Hi +1 to drop bucketing sink FLIP-115 also needs to be prioritized for 1.11 在 2020/3/13 上午10:07,“Jingsong Li” 写入: Hi Robert, +1 to drop it but maybe not 1.11. ORC has not been supported on StreamingFileSink. I have seen lots of users run ORC in the bucketing sink.

Re: [DISCUSS] Drop Bucketing Sink

2020-03-12 Thread Jingsong Li
Hi Robert, +1 to drop it but maybe not 1.11. ORC has not been supported on StreamingFileSink. I have seen lots of users run ORC in the bucketing sink. Best, Jingsong Lee On Fri, Mar 13, 2020 at 1:11 AM Seth Wiesman wrote: > Sorry, I meant FLIP-46. > > Seth > > On Thu, Mar 12, 2020 at 11:52 AM

Re: [DISCUSS] Drop Bucketing Sink

2020-03-12 Thread Seth Wiesman
Sorry, I meant FLIP-46. Seth On Thu, Mar 12, 2020 at 11:52 AM Seth Wiesman wrote: > I agree with David, I think FLIP-49 needs to be prioritized for 1.11 if we > want to drop the bucketing sink. > > Seth > > On Thu, Mar 12, 2020 at 10:53 AM David Anderson > wrote: > >> The BucketingSink is stil

Re: [DISCUSS] Drop Bucketing Sink

2020-03-12 Thread Seth Wiesman
I agree with David, I think FLIP-49 needs to be prioritized for 1.11 if we want to drop the bucketing sink. Seth On Thu, Mar 12, 2020 at 10:53 AM David Anderson wrote: > The BucketingSink is still somewhat widely used, I think in part because of > shortcomings in the StreamingFileSink. > > I wo

Re: [DISCUSS] Drop Bucketing Sink

2020-03-12 Thread David Anderson
The BucketingSink is still somewhat widely used, I think in part because of shortcomings in the StreamingFileSink. I would hope that in tandem with removing the bucketing sink we could also address some of these issues. I'm thinking in particular of issues that are waiting on FLIP-46 [1]. Removin

Re: [DISCUSS] Drop Bucketing Sink

2020-03-12 Thread Zhijiang
Thanks for driving this discussion, Robert! This e2e test really fails frequently. +1 to drop bucketing sink, it is not worth paying more efforts since deprecated. Best, Zhijiang -- From:Jeff Zhang Send Time:2020 Mar. 12 (Thu.)

Re: [DISCUSS] Drop Bucketing Sink

2020-03-12 Thread Hequn Cheng
Good idea! +1 for dropping the BucketingSink. Best, Hequn > On Mar 12, 2020, at 10:40 PM, Robert Metzger wrote: > > Hi all, > > I'm currently investigating a failing end to end test for the bucketing > sink [1]. > The bucketing sink has been deprecated in the 1.9 release [2], because we > have

Re: [DISCUSS] Drop Bucketing Sink

2020-03-12 Thread Aljoscha Krettek
+1 Aljoscha

Re: [DISCUSS] Drop Bucketing Sink

2020-03-12 Thread Jeff Zhang
+1, dropping deprecated api is always necessary for a sustainable project. Kostas Kloudas 于2020年3月12日周四 下午11:06写道: > Hi Robert, > > +1 for dropping the BucketingSink. > In any case, it has not been maintained for quite some time. > > Cheers, > Kostas > > On Thu, Mar 12, 2020 at 3:41 PM Robert Me

Re: [DISCUSS] Drop Bucketing Sink

2020-03-12 Thread Kostas Kloudas
Hi Robert, +1 for dropping the BucketingSink. In any case, it has not been maintained for quite some time. Cheers, Kostas On Thu, Mar 12, 2020 at 3:41 PM Robert Metzger wrote: > > Hi all, > > I'm currently investigating a failing end to end test for the bucketing > sink [1]. > The bucketing sin