Kafka Offset Commit

2021-08-10 Thread Talat Uyarer
Hi Samza Community, This is my first email. Forgive my lack of knowledge about samza. I am running a testing job in my environment. I run in local model but somehow my job is processing data however it does not commit offset on Kafka side. I use an apache beam samza runner. My pipeline is simply

Re: Kafka Offset Commit

2021-08-10 Thread rayman preet
Hi Talat, Since in the job.properties the task.checkpoint.factory is set to FileSystemCheckpointManagerFactory and not org.apache.samza.checkpoint.kafka.KafkaCheckpointManagerFactory. That is why its writing checkpoints to the filesystem (with its. location controlled by task.checkpoint.path). ht

Re: Kafka Offset Commit

2021-08-10 Thread Talat Uyarer
Thank you rayman. But my question is when i check kafka consumer group of the job. I dont see any offset movement. I chose to store checkpoints on file system. Do you think because of that i dont see my job's consumer group does not move offset ? On Tue, Aug 10, 2021, 9:32 PM rayman preet wrote