Re: Using the BucketingSink with Flink 1.4.0

2018-03-09 Thread lrao
Thank you for your responses Stephan and Piotrek! It's great to know that the hadoop-free Bucketing Sink might be available as early as 1.5.x! In the meantime, I have been trying workarounds but I am currently facing issues making it work. I tried including my Hadoop dependencies only in my use

Re: Using the BucketingSink with Flink 1.4.0

2018-03-09 Thread Stephan Ewen
Hi! Yes, the bucketing sink is unfortunately still tied to some specific Hadoop file systems, due to a special way of using truncate() and append(). This is very high up our list post the 1.5 release, possibly even backportable to 1.5.x. The plan is to create a new Bucketing Sink based on Flink'

Re: Using the BucketingSink with Flink 1.4.0

2018-03-09 Thread Piotr Nowojski
Hi, There is an quite old ticket about this issue. Feel free to bump it in the comment to rise it’s priority. https://issues.apache.org/jira/browse/FLINK-5789 Regarding a walk around, maybe someone else will know more. There was a similar dis

Using the BucketingSink with Flink 1.4.0

2018-03-08 Thread lrao
I want to use the BucketingSink in the hadoop-free Flink system (i.e. 1.4.0) but currently I am kind of blocked because of its dependency on the Hadoop file system. 1. Is this something that's going to be fixed in the next version of Flink? 2. In the meantime, to unblock myself, what is the bes