Re: A question about checkpointing

2017-05-17 Thread 舒琦
Hi Yi, I got it now, thanks for your help! Qi Shu > 在 2017年5月18日,05:22,Yi Pan 写道: > > Hi, Qi, > > This would depend on the following two factors: > # whether the send() is async or sync > # how do you handle the send failure > > If the send() is sync, you will always receive

Re: A question about checkpointing

2017-05-17 Thread Yi Pan
Hi, Qi, This would depend on the following two factors: # whether the send() is async or sync # how do you handle the send failure If the send() is sync, you will always receive an exception in your process() method when MessageCollector.send() is invoked. Hence, if your code does not handle the

Re: A question about checkpointing

2017-05-11 Thread 舒琦
Hi Jagadish, I may not express my questions clearly. Here is what I want to know. When MessageCollector.send is called in process method, if sending fail and fail again, under this situation is it possible to cause data loss ( continue to fetch and process messages, but can’t s

Re: A question about checkpointing

2017-05-11 Thread Jagadish Venkatraman
Hi Qi, >> If one record can’t be sent out all the time, then the consumer will still fetch messages or not, and what about the offset checkpointing? Polling / fetching messages from the consumer (in case of Kafka) happens in a separate thread. Samza offers an at-least once processing gua

A question about checkpointing

2017-05-11 Thread 舒琦
Hi, Below is the description about checkpointing. 『Checkpointing is guaranteed to only cover events that are fully processed. It happens only when there are no pending process()/processAsync() or WindowableTask.window() invocations. All preceding invocations happen-before check