Re: Flink Cluster Load Distribution Question

2016-09-13 Thread Robert Metzger
Hi Amir, I would recommend to post such questions to the user@flink mailing list in the future. This list is meant for development-related topics. I think we need more details to understand why your application is not running properly. Can you quickly describe what your topology is doing? Are you

Re: Savepoint and restart in the same command?

2016-09-13 Thread Konstantin Knauf
Hi Ron, I think, Till mentioned in his talk on re-scaling streaming jobs, that a "Savepoint and Stop" command is one of the upcoming features in the "dynamic scaling" thread. Cheers, Konstantin On 12.09.2016 14:39, Ron Crocker wrote: > I attended Jamie Grier’s excellent presentation at Flink Fo

[jira] [Created] (FLINK-4615) Reusing the memory allocated for the drivers and iterators

2016-09-13 Thread ramkrishna.s.vasudevan (JIRA)
ramkrishna.s.vasudevan created FLINK-4615: - Summary: Reusing the memory allocated for the drivers and iterators Key: FLINK-4615 URL: https://issues.apache.org/jira/browse/FLINK-4615 Project: Fl

[jira] [Created] (FLINK-4616) Kafka consumer doesn't store last emmited watermarks per partition in state

2016-09-13 Thread Yuri Makhno (JIRA)
Yuri Makhno created FLINK-4616: -- Summary: Kafka consumer doesn't store last emmited watermarks per partition in state Key: FLINK-4616 URL: https://issues.apache.org/jira/browse/FLINK-4616 Project: Flink

Re: checkpointing a OneInputStreamOperator/AbstractStreamOperator

2016-09-13 Thread dan bress
Aljoscha, Thanks. I originally looked at the keyed state streaming. The problem I have with this approach is that it requires that I manage which keys this operator has seen and set them prior to getting or retrieving state, correct? My operator is used on a keyed stream. This means I have t

[jira] [Created] (FLINK-4617) Kafka & Flink duplicate messages on restart

2016-09-13 Thread Matthew Barlocker (JIRA)
Matthew Barlocker created FLINK-4617: Summary: Kafka & Flink duplicate messages on restart Key: FLINK-4617 URL: https://issues.apache.org/jira/browse/FLINK-4617 Project: Flink Issue Type:

Re: Flink Cluster Load Distribution Question

2016-09-13 Thread amir bahmanyari
Hi Robert,I forwarded my Qs to the users forum...No response ...Is there anyway I can get some answers to my questions pls?I really really appreciate your help...CheersAmir- From: Robert Metzger To: "dev@flink.apache.org" ; amir bahmanyari Sent: Tuesday, September 13, 2016 1:15 AM S

[DISCUSS] Proposal for Asynchronous I/O in FLINK

2016-09-13 Thread Jeffery David
Hi ALL, Recently, we have designed and implemented a new proposal in FLINK to support Asynchronous Operation while streaming. The main feature of this proposal is to introduce async i/o operation in FLINK to boost the TPS of streaming job without delaying the checkpoint, and provide an easy way fo

[DISCUSS] Proposal for Asynchronous I/O in FLINK

2016-09-13 Thread Jeffery David
Hi ALL, Recently, we have designed and implemented a proposal in FLINK to support Asynchronous Operation while streaming. The main feature of this proposal is to introduce async i/o operation in FLINK to boost the TPS of streaming job without delaying the checkpoint, and provide an easy way for th