Hi Srikanth, I am not exactly sure if I understand your question correctly.
One way to track the progress is to get the current record offset (you can obtain it in the low lever Processor API via the provided Context object). Otherwise, on commit, all writes to intermediate topics are flushed to Kafka and the source offsets get committed to Kafka, too. A KafkaStream application internally uses the standard high level Java KafkaConsumer (all instances of a single application belong to the same consumer group) and standard Java KafkaProducer. So you can use standard Kafka tools to access this information. Does this answer your question? -Matthias On 05/31/2016 09:10 PM, Srikanth wrote: > Hi, > > How can I track the progress of a kafka streaming job? > The only reference I see is "commit.interval.ms" which controls how often > offset is committed. > By default where is it committed and is there a tool to read it back? May > be something similar to bin/kafka-consumer-groups.sh. > > I'd like to look at details for source & intermediate topics too. > > Srikanth >
signature.asc
Description: OpenPGP digital signature
