Re: samza job start takes 20 minutes to figure out the Checkpointed offset

2016-05-08 Thread Liu Bo
Thanks for the reply Jagadish, it's really a good point. I check the kafka configuration, log compaction is enabled, but other compaction related settings are all default at the server side. The checkpoint topic has a topic level config which set the cleanup policy to compact (done by samza while

Re: Can a job write to output topic that is also input topic

2016-05-08 Thread Jagadish Venkatraman
Hey Andy, Afaik, Samza does not stop you from doing that. However, you should take extra care to avoid cycles / loops if you do that in the same job. It may also make debugging your logic harder for you. On Sunday, May 8, 2016, Andy Chambers wrote: > In designing a samza job, we have come up wit

Can a job write to output topic that is also input topic

2016-05-08 Thread Andy Chambers
In designing a samza job, we have come up with a situation where one of the output topics would also be an input topic (it's a bit of a state machine and we're using this topic as a changelog topic for the state machine). Is this use-case supported? Cheers, Andy Andy Chambers | Full-Stack Software