Re: Use Single Sink For All windows

2017-06-13 Thread Aljoscha Krettek
not be called. > On 12. Jun 2017, at 19:12, rhashmi wrote: > > I think CheckpointListener? > > > > > -- > View this message in context: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Use-Single-Sink-For-All-windows-tp13475p13653.html >

Re: Use Single Sink For All windows

2017-06-12 Thread rhashmi
I think CheckpointListener? -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Use-Single-Sink-For-All-windows-tp13475p13653.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: Use Single Sink For All windows

2017-06-12 Thread rhashmi
stream. With this approach we can assure that we update state only if the checkpoint was successful. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Use-Single-Sink-For-All-windows-tp13475p13652.html Sent from the Apache Flink User Mailing Lis

Re: Use Single Sink For All windows

2017-06-12 Thread Aljoscha Krettek
Ah, I think now I get your problem. You could manually implement batching inside your SinkFunction, The SinkFunction would batch values in memory and periodically (based on the count of values and on a timeout) send these values as a single batch to MySQL. To ensure that data is not lost you can

Re: Use Single Sink For All windows

2017-06-08 Thread Nico Kruber
How about using asynchronous I/O operations? https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/ asyncio.html Nico On Tuesday, 6 June 2017 16:22:31 CEST rhashmi wrote: > because of parallelism i am seeing db contention. Wondering if i can merge > sink of multiple windows and

Re: Use Single Sink For All windows

2017-06-06 Thread rhashmi
because of parallelism i am seeing db contention. Wondering if i can merge sink of multiple windows and insert in batch. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Use-Single-Sink-For-All-windows-tp13475p13525.html Sent from the Apache

Re: Use Single Sink For All windows

2017-06-06 Thread Aljoscha Krettek
p://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Use-Single-Sink-For-All-windows-tp13475.html > Sent from the Apache Flink User Mailing List archive. mailing list archive at > Nabble.com.

Use Single Sink For All windows

2017-06-04 Thread rhashmi
://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Use-Single-Sink-For-All-windows-tp13475.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.