Re: Difference between BucketingSink and StreamingFileSink

2018-10-04 Thread Aljoscha Krettek
No worries! :-) it's nice that you also posted the solution > On 4. Oct 2018, at 13:23, Averell wrote: > > Hi, > > Sorry for wasting your time. I found the solution for that question > regarding event-time: a class that extends BucketAssigner would do the > needful: > > class SdcTimeBucketAssi

Re: Difference between BucketingSink and StreamingFileSink

2018-10-04 Thread Averell
Hi, Sorry for wasting your time. I found the solution for that question regarding event-time: a class that extends BucketAssigner would do the needful: class SdcTimeBucketAssigner[T <: MyClass](prefix: String, formatString: String) extends BucketAssigner[T, String]{ @transient var

Re: Difference between BucketingSink and StreamingFileSink

2018-10-04 Thread Averell
Hi, https://issues.apache.org/jira/browse/FLINK-9749 <<< as per this ticket, StreamingFileSink is a newer option, which is better than BucketingSink for Parquet. Would love to see some example one using that. Thanks and best regards, Averell -- Sent from: http://apache-flink-user-mailing-list-

Difference between BucketingSink and StreamingFileSink

2018-10-04 Thread Averell
Hi everyone, I am trying to persist my stream into parquet files. In the documents, I can see two different file sinks: BucketingSink (Rolling File Sink) and StreamingFileSink. I could not see any information regarding the differences between these two types. Which one should I choose for writing