Thanks Matthias!
1) I didn't realize kafka-consumer-groups.sh only queries consumer
coordinator.
I was checking after terminating the streaming app. Got this via
console-consumer.
2) Understood.
3) Nope. Will check this out.
4)Yes, I can probably have a processorSupplier for this.
Srikanth
On
Srikanth,
KafkaStreams uses the new consumers, thus you need to use
> bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server localhost:9092
> --list
instead of "--zookeeper XXX"
AbstractTask.commit() is called each "commit.interval.ms". A task is
requested to persist all buffered data
Matthias,
"""bin/kafka-consumer-groups.sh --zookeeper localhost:2181/kafka10
--list""" output didn't show the group I used in streams app.
Also, AbstractTask.java had a commit() API. That made me wonder if offset
management was overridden too.
I'm trying out KafkaStreams for one new streaming app
Hi Srikanth,
I am not exactly sure if I understand your question correctly.
One way to track the progress is to get the current record offset (you
can obtain it in the low lever Processor API via the provided Context
object).
Otherwise, on commit, all writes to intermediate topics are flushed to
Hi,
How can I track the progress of a kafka streaming job?
The only reference I see is "commit.interval.ms" which controls how often
offset is committed.
By default where is it committed and is there a tool to read it back? May
be something similar to bin/kafka-consumer-groups.sh.
I'd like to loo
