Strange. I am unable to comment on the behavior because I don't know what
your checkpoints looked like in the checkpoint topic.
Did you try reading the checkpoint topic log ?
If you setting systems.kafka.streams.nogoalids.samza.reset.offset = true,
you are essentially ignoring checkpoints for tha
I'm trying to debug our samza job, which seem to be stuck from consuming
from our Kafka stream.
Every time I redeploy the job, only the same handful of events get
consumed, and then no more events get processed. I manually checked to make
sure the input stream is live and flowing. I also tried bot
Hi David,
Appreciate the feedback. Currently, Samza is single-threaded. The process()
callback executes exactly in the context of the same thread that processes
events. You can take a process dump (often over a period of time
repeatedly) and figure out where the threads are stuck to debug these ki
HI David,
This configuration you have tweaked
(systems.kafka.consumer.auto.offset.reset) is honored only when one of the
following condition holds:
* topic doesn't exist
* checkpoint is older than the maximum message history retained by the
brokers
So, my questions are :
Are you using checkpointin
Hi, I wanna resurface this thread because I'm still facing issues with our
samza not receiving events.
Our samza job metric "SamzaContainerMetrics.process-calls" dropped to zero
today again. So does "SamzaContainerMetrics.process-envelopes" (of course).
Current topic offset and task checkpoint rev
Finally seeing events flowing again.
Yes, the "systems.kafka.consumer.auto.offset.reset" option is probably not
a factor here. And yes, I am using checkpointing (kafka). Not sure if the
offsets are messed up. But I was able to use
"systems.kafka.streams.nogoalids.samza.reset.offset=true" to reset
Looks like this has nothing to do with checkpointing. Our samza job has an
issue communicating an external service, which left the particular
process() call waiting indefinitely. And it doesn't look like samza has a
way to timeout a processing cycle.
On Thu, Mar 17, 2016 at 5:42 PM, David Yu wrot
No, instead, I updated the checkpoint topic with the "upcoming" offsets. (I
should have done a check before that though).
So a related question: if I delete the checkpoint topic from Kafka, that
would essentially clear up all the offset info and samza will be able to
recreate this topic with the l
Strangely, I was not able to get checkpoint value for one particular
partition. Could this cause the job to be stuck?
On Thu, Mar 17, 2016 at 5:23 PM, David Yu wrote:
> Hi, I wanna resurface this thread because I'm still facing issues with our
> samza not receiving events.
>
> Our samza job metr
