Matthias, """bin/kafka-consumer-groups.sh --zookeeper localhost:2181/kafka10 --list""" output didn't show the group I used in streams app. Also, AbstractTask.java had a commit() API. That made me wonder if offset management was overridden too.
I'm trying out KafkaStreams for one new streaming app we are working on. We'll most likely stick to DSL for that. Does the DSL expose any stat or debug info? Or any way to access the underlying Context? Srikanth On Thu, Jun 2, 2016 at 9:30 AM, Matthias J. Sax <matth...@confluent.io> wrote: > Hi Srikanth, > > I am not exactly sure if I understand your question correctly. > > One way to track the progress is to get the current record offset (you > can obtain it in the low lever Processor API via the provided Context > object). > > Otherwise, on commit, all writes to intermediate topics are flushed to > Kafka and the source offsets get committed to Kafka, too. > > A KafkaStream application internally uses the standard high level Java > KafkaConsumer (all instances of a single application belong to the same > consumer group) and standard Java KafkaProducer. > > So you can use standard Kafka tools to access this information. > > Does this answer your question? > > -Matthias > > On 05/31/2016 09:10 PM, Srikanth wrote: > > Hi, > > > > How can I track the progress of a kafka streaming job? > > The only reference I see is "commit.interval.ms" which controls how > often > > offset is committed. > > By default where is it committed and is there a tool to read it back? May > > be something similar to bin/kafka-consumer-groups.sh. > > > > I'd like to look at details for source & intermediate topics too. > > > > Srikanth > > > >
