Re: Question about flink message processing guarantee

2015-12-02 Thread Stephan Ewen
There is an overview of what guarantees what sources can give you: https://ci.apache.org/projects/flink/flink-docs-master/apis/fault_tolerance.html#fault-tolerance-guarantees-of-data-sources-and-sinks On Wed, Dec 2, 2015 at 9:19 AM, Till Rohrmann wrote: > Just a small addition. Your sources have

Re: Question about flink message processing guarantee

2015-12-02 Thread Till Rohrmann
Just a small addition. Your sources have to be replayable to some extent. With replayable I mean that they can continue from some kind of offset. Otherwise the check pointing won't help you. The Kafka source supports that for example. Cheers, Till On Dec 1, 2015 11:55 PM, "Márton Balassi" wrote:

Re: Question about flink message processing guarantee

2015-12-01 Thread Márton Balassi
Dear Jerry, If you do not enable checkpointing you get the at most once processing guarantee (some might call that no guarantee at all). When you enable checkpointing you can choose between exactly and at least once semantics. The latter provides better latency. Best, Marton On Tue, Dec 1, 2015

Question about flink message processing guarantee

2015-12-01 Thread Jerry Peng
Hello, I have a question regarding link streaming. I now if you enable checkpointing you can have exactly once processing guarantee. If I do not enable checkpointing what are the semantics of the processing? At least once? Best, Jerry