I have a strange issue. We just replaced failed zookeeper server. We killed the job in the interim and resubmitted the job to yarn. The events were getting processed but at some point, the events stopped processing. I have checked RM and application logs but they seem to be fine.
We then restarted Kafka and it seem to have fixed the issue. I have checked into Kafka logs. Apart from some warn messages, there were no big red flags either. We are using Kafka 0.8 and Samza 0.8. One of the things I did to test was to use the console producer and produced a raw message. I am sure this could not have caused any issues. I am not sure how to debug this issue. Appreciate any pointers. Thanks, Shekar